Eliminating Duplicates from a List in Hugo

Recently I was finishing up converting a site from having a LAMP back-end to a Hugo static site. I exported all the MYSQL tables to JSON and stored them away in the data folder.

When processing these JSON records, any column which had a one-to-many relationship would exist more than once in any listing for the “many” side. So I needed to find a way to remove duplicate elements in a list. Here’s how I did it.

Process Overview

The overall process was this:

1. Create a shortcode to render most, if not all, content for the page. The shortcode would get ID as an argument and use that information to process the JSON data and render the calling page.
2. Create a separate markdown page with the shortcode for all the items which had an ID.

But this post isn’t about the shortcode. We’ll cover that in another post. I just wanted to offer up a high-level view of how this code would actually be put to use.

First, create an empty slice

{{ $s := slice }}

The empty slice will become our final list with no duplicate ID elements.

Now range through the elements and add the ID to the empty slice

{{ range $results }}
 {{ $s = $s | append .rider_id }}
{{end }}

And finally, run Hugo’s uniq collection function

{{ $uniques := collection.Uniq $s }}
or
{{ $uniques := uniq $s }}

All done. Simple. Here’s the code all in one place

{{ $s := slice }}
{{ range $results }}
 {{ $s = $s | append .rider_id }}
{{end }}
{{ $uniques := uniq $s }}