JavaScript: Removing array duplicates

Let’s take a look at some of the ways we can effectively remove duplicates from JavaScript arrays with various degrees of execution and complexity.

filter()

This method creates a new array that only consists of elements of the starting array that pass a certain condition. Here we use another built-in method of JavaScript arrays, indexOf() that returns the first index where it found our provided element in the array. By doing this we simply check if our element is already somewhere in the array and not in the current iteration position, i, which means that the element is a duplicate:

const animals = ["Dog", "Cat", "Mouse", "Horse", "Dog", "Rhino"];

const uniqueAnimals = animals.filter(
  (animal, i) => animals.indexOf(animal) === i
); // ["Dog", "Cat", "Mouse", "Horse", "Rhino"]

reduce()

With reduce, we apply a function to each element of the array and reduce it to a new single value. Our starting single value is an empty array to which we add elements that pass our filter. We can reduce our array to a new one by simply checking if the current element has already been added to the new array using the includes() array method:

const animals = ["Dog", "Cat", "Mouse", "Horse", "Dog", "Rhino"];

const uniqueAnimals = animals.reduce((newAnimals, currentAnimal) => {
  if (!newAnimals.includes(currentAnimal)) {
    newAnimals.push(currentAnimal);
  }
  return newAnimals;
}, []); // ["Dog", "Cat", "Mouse", "Horse", "Rhino"]

forEach()

If we are dealing with large arrays we want to use a hashtable rather that includes() or indexOf() when checking if our element already exists as it gives us linear time. Our hashtable is a Map() and its keys are the animals. First, we check if the key exists, if not we add it with the value true ensuring that the next animal with the same name will be skipped by the former if-clause:

const animals = ["Dog", "Cat", "Mouse", "Horse", "Dog", "Rhino"];

const uniqueAnimalsHashtable = new Map();

animals.forEach(animal => {
  if (!uniqueAnimalsHashtable[animal]) {
    uniqueAnimalsHashtable[animal] = true;
  }
});

const uniqueAnimals = Object.keys(uniqueAnimalsHashtable); // ["Dog", "Cat", "Mouse", "Horse", "Rhino"]

Set object

Set is a new ES6 feature, an object that lets you store only unique values. So if we pass it an array it will automatically remove the duplicates:

const animals = ["Dog", "Cat", "Mouse", "Horse", "Dog", "Rhino"];

const animalsSet = new Set(animals); // {"Dog", "Cat", "Mouse", "Horse", "Rhino"}

All we have to do next is revert the set back to an array in one of the following ways:

const uniqueAnimals1 = [...animalsSet]; // ["Dog", "Cat", "Mouse", "Horse", "Rhino"]

const uniqueAnimals2 = Array.from(animalsSet); // ["Dog", "Cat", "Mouse", "Horse", "Rhino"]

Or use a one-liner:

const uniqueAnimals = [...new Set(animals)];

Launching a new JavaScript project? Need help on an existing project? Work with us