74

I have a JavaScript array like this:

var myData=['237','124','255','124','366','255'];

I need the array elements to be unique and sorted:

myData[0]='124';
myData[1]='237';
myData[2]='255';
myData[3]='366';

Even though the members of array look like integers, they're not integers, since I have already converted each to be string:

var myData[0]=num.toString();
//...and so on.

Is there any way to do all of these tasks in JavaScript?

19 Answers 19

157

This is actually very simple. It is much easier to find unique values, if the values are sorted first:

function sort_unique(arr) {
  if (arr.length === 0) return arr;
  arr = arr.sort(function (a, b) { return a*1 - b*1; });
  var ret = [arr[0]];
  for (var i = 1; i < arr.length; i++) { //Start loop at 1: arr[0] can never be a duplicate
    if (arr[i-1] !== arr[i]) {
      ret.push(arr[i]);
    }
  }
  return ret;
}
console.log(sort_unique(['237','124','255','124','366','255']));
//["124", "237", "255", "366"]
Sign up to request clarification or add additional context in comments.

14 Comments

What's the point of doing a*1 - b*1 instead of just a - b?
After 9 failed attempts, Randall Munroe's StackSort sent me here, great example :P gkoberger.github.io/stacksort
@FiHoran It seems to come here every time!
Took it 15 attempts, but StackSort got me here.
A 7 year old PR was just merged for the stacksort site thanks to a thread on Reddit. Here come the upvotes - it's going to be sizzling hot again. github.com/gkoberger/stacksort/pull/4
|
62

You can now achieve the result in just one line of code.

Using new Set to reduce the array to unique set of values. Apply the sort method after to order the string values.

var myData=['237','124','255','124','366','255']

var uniqueAndSorted = [...new Set(myData)].sort() 

UPDATED for newer methods introduced in JavaScript since time of question.

1 Comment

For when it matters, this is likely not very efficient as you have to do the n log n twice.
43

This might be adequate in circumstances where you can't define the function in advance (like in a bookmarklet):

myData.sort().filter(function(el,i,a){return i===a.indexOf(el)})

6 Comments

myData.sort().filter(function(el,i,a){return i==a.indexOf(el);})
myData.sort().filter(function(el,i,a){return !i||el!=a[i-1];})
Case-insensitive: myData.sort().filter(function(el,i,a){return !i||el.toLowerCase()!=a[i-1].toLowerCase();})
myData.sort().filter((x,i,a)=>x!=a[i-1]) (this will remove the first null value but otherwise works great)
ES6 syntax: arr.sort().filter((el,i,a) => (i===a.indexOf(el)));
|
22

Here's my (more modern) approach using Array.protoype.reduce():

[2, 1, 2, 3].reduce((a, x) => a.includes(x) ? a : [...a, x], []).sort()
// returns [1, 2, 3]

Edit: More performant version as pointed out in the comments:

arr.sort().filter((x, i, a) => !i || x != a[i-1])

1 Comment

This is an okay selection, but slow when the array is large - O(n^2) for the reduce because you call indexOf for every element, then O(n log n) for the sort at the end. Once you've sorted it you only need linear time to remove duplicates :)
17
function sort_unique(arr) {
    return arr.sort().filter(function(el,i,a) {
        return (i==a.indexOf(el));
    });
}

1 Comment

filter isn’t jQuery either. That would be $.filter.
10

How about:

array.sort().filter(function(elem, index, arr) {
  return index == arr.length - 1 || arr[index + 1] != elem
})

This is similar to @loostro answer but instead of using indexOf which will reiterate the array for each element to verify that is the first found, it just checks that the next element is different than the current.

2 Comments

This should have been the accepted answer. Simple, elegant, native, bulletproof. The others with O(n^2), external libraries, extra functions and array copying... Am I missing something, why this shoudn't be the best answer?
This code doesn't work as expected, it doesn't sort correctly single digit integers but it works ok if inside sort() you also add function(a,b) { return a - b; } codepen.io/ivan-topi/pen/BaybgPB
5

Try using an external library like underscore

var f = _.compose(_.uniq, function(array) {
    return _.sortBy(array, _.identity);
});

var sortedUnique = f(array);

This relies on _.compose, _.uniq, _.sortBy, _.identity

See live example

What is it doing?

We want a function that takes an array and then returns a sorted array with the non-unique entries removed. This function needs to do two things, sorting and making the array unique.

This is a good job for composition, so we compose the unique & sort function together. _.uniq can just be applied on the array with one argument so it's just passed to _.compose

the _.sortBy function needs a sorting conditional functional. it expects a function that returns a value and the array will be sorted on that value. Since the value that we are ordering it by is the value in the array we can just pass the _.identity function.

We now have a composition of a function that (takes an array and returns a unique array) and a function that (takes an array and returns a sorted array, sorted by their values).

We simply apply the composition on the array and we have our uniquely sorted array.

Comments

5

This function doesn't fail for more than two duplicates values:

function unique(arr) {
    var a = [];
    var l = arr.length;
    for(var i=0; i<l; i++) {
        for(var j=i+1; j<l; j++) {
            // If a[i] is found later in the array
            if (arr[i] === arr[j])
              j = ++i;
        }
        a.push(arr[i]);
    }
    return a;
};

Comments

4

Here is a simple one liner with O(N), no complicated loops necessary.

> Object.keys(['a', 'b', 'a'].reduce((l, r) => l[r] = l, {})).sort()
[ 'a', 'b' ]

Explanation

Original data set, assume its coming in from an external function

const data = ['a', 'b', 'a']

We want to group all the values onto an object as keys as the method of deduplication. So we use reduce with an object as the default value:

[].reduce(fn, {})

The next step is to create a reduce function which will put the values in the array onto the object. The end result is an object with a unique set of keys.

const reduced = data.reduce((l, r) => l[r] = l, {})

We set l[r] = l because in javascript the value of the assignment expression is returned when an assignment statement is used as an expression. l is the accumulator object and r is the key value. You can also use Object.assign(l, { [r]: (l[r] || 0) + 1 }) or something similar to get the count of each value if that was important to you.

Next we want to get the keys of that object

const keys = Object.keys(reduced)

Then simply use the built-in sort

console.log(keys.sort())

Which is the set of unique values of the original array, sorted

['a', 'b']

Comments

3

The solution in a more elegant way.

var myData=['237','124','255','124','366','255'];

console.log(Array.from(new Set(myData)).sort((a,b) => a - b));

I know the question is very old, but maybe someone will come in handy

1 Comment

This doesn't actually seem to reliably work, e.g. Array.from(new Set([2,2,3,2,3,4,5,6,7,8,5,6,7,8,9,5,6,7,8,9,10,5,6,7,8,9,10,11,5,6,7,8,9,10,11,12])).sort() returns [10,11,12,2,3,4,5,6,7,8,9].
1

A way to use a custom sort function

//func has to return 0 in the case in which they are equal
sort_unique = function(arr,func) {
        func = func || function (a, b) {
            return a*1 - b*1;
        };
        arr = arr.sort(func);
        var ret = [arr[0]];
        for (var i = 1; i < arr.length; i++) {
            if (func(arr[i-1],arr[i]) != 0) 
                ret.push(arr[i]);
            }
        }
        return ret;
    }

Example: desc order for an array of objects

MyArray = sort_unique(MyArray , function(a,b){
            return  b.iterator_internal*1 - a.iterator_internal*1;
        });

Comments

1

No redundant "return" array, no ECMA5 built-ins (I'm pretty sure!) and simple to read.

function removeDuplicates(target_array) {
    target_array.sort();
    var i = 0;

    while(i < target_array.length) {
        if(target_array[i] === target_array[i+1]) {
            target_array.splice(i+1,1);
        }
        else {
            i += 1;
        }
    }
    return target_array;
}

Comments

0

I guess I'll post this answer for some variety. This technique for purging duplicates is something I picked up on for a project in Flash I'm currently working on about a month or so ago.

What you do is make an object and fill it with both a key and a value utilizing each array item. Since duplicate keys are discarded, duplicates are removed.

var nums = [1, 1, 2, 3, 3, 4, 5, 5, 6, 7, 7, 8, 9, 9, 10];
var newNums = purgeArray(nums);

function purgeArray(ar)
{
    var obj = {};
    var temp = [];
    for(var i=0;i<ar.length;i++)
    {
        obj[ar[i]] = ar[i];
    }
    for (var item in obj)
    {
        temp.push(obj[item]);
    }
    return temp;
}

There's already 5 other answers, so I don't see a need to post a sorting function.

2 Comments

I was going to do a sort function, but I lazily left it out before giving up when an answer was accepted. I'll edit it quickly.
Can this function be extended for objects rather than primitives?
0
// Another way, that does not rearrange the original Array 
// and spends a little less time handling duplicates.

function uniqueSort(arr, sortby){
    var A1= arr.slice();
    A1= typeof sortby== 'function'? A1.sort(sortby): A1.sort();

    var last= A1.shift(), next, A2= [last];
    while(A1.length){
        next= A1.shift();
        while(next=== last) next= A1.shift();
        if(next!=undefined){
            A2[A2.length]= next;
            last= next;
        }
    }
    return A2;
}
var myData= ['237','124','255','124','366','255','100','1000'];
uniqueSort(myData,function(a,b){return a-b})

// the ordinary sort() returns the same array as the number sort here,
// but some strings of digits do not sort so nicely numerical.

Comments

0

function sort() only is only good if your number has same digit, example:

var myData = ["3","11","1","2"]

will return;

var myData = ["1","11","2","3"]

and here improvement for function from mrmonkington

myData.sort().sort(function(a,b){return a - b;}).filter(function(el,i,a){if(i==a.indexOf(el) & el.length>0)return 1;return 0;})

the above function will also delete empty array and you can checkout the demo below

http://jsbin.com/ahojip/2/edit

1 Comment

Why are you sorting it twice? What’s with the & el.length > 0?
0

O[N^2] solutions are bad, especially when the data is already sorted, there is no need to do two nested loops for removing duplicates. One loop and comparing to the previous element will work great.

A simple solution with O[] of sort() would suffice. My solution is:

function sortUnique(arr, compareFunction) {
  let sorted = arr.sort(compareFunction);
  let result = sorted.filter(compareFunction
    ? function(val, i, a) { return (i == 0 || compareFunction(a[i-1], val) != 0); }
    : function(val, i, a) { return (i == 0 || a[i-1] !== val); }
  );
  return result;
}

BTW, can do something like this to have Array.sortUnique() method:

Array.prototype.sortUnique = function(compareFunction) {return sortUnique(this, compareFunction); }

Furthermore, sort() could be modified to remove second element if compare() function returns 0 (equal elements), though that code can become messy (need to revise loop boundaries in the flight). Besides, I stay away from making my own sort() functions in interpreted languages, since it will most certainly degrade the performance. So this addition is for the ECMA 2019+ consideration.

Comments

0

The fastest and simpleness way to do this task.

const N = Math.pow(8, 8)
let data = Array.from({length:  N}, () => Math.floor(Math.random() * N))
let newData = {}
let len = data.length

// the magic
while (len--) {
    newData[data[len]] = true
}

Comments

0

var array = [2,5,4,2,5,9,4,2,6,9,0,5,4,7,8];

var unique_array = [...new Set(array)]; // [ 2, 5, 4, 9, 6, 0, 7, 8 ]

var uniqueWithSorted = unique_array.sort();

console.log(uniqueWithSorted);
output = [ 0, 2, 4, 5, 6, 7, 8, 9 ]

Here, we used only Set for removing duplicity from the array and then used sort for sorting array in ascending order.

Comments

-3

I'm afraid you can't combine these functions, ie. you gotta do something like this:-

myData.unique().sort();

Alternatively you can implement a kind of sortedset (as available in other languages) - which carries both the notion of sorting and removing duplicates, as you require.

Hope this helps.

References:-

Array.sort

Array.unique

3 Comments

My Chrome doesn't know this method.
Maybe you should have specified that the .unique() you are suggesting is not a native function, it has to be defined (it's said in the link you posted anyway).
I agree, it would make things a lot more practical, but perhaps makes things too easy??

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.