Lodash remove duplicates from array

Question

This is my data:

[
    {
        url: 'www.example.com/hello',
        id: "22"    
    },
    {
        url: 'www.example.com/hello',
        id: "22"    
    },
    {
        url: 'www.example.com/hello-how-are-you',
        id: "23"    
    },
    {
        url: 'www.example.com/i-like-cats',
        id: "24"    
    },
    {
        url: 'www.example.com/i-like-pie',
        id: "25"    
    }
]

With Lodash, how could I remove objects with duplicate id keys? Something with filter, map and unique, but not quite sure.

My real data set is much larger and has more keys, but the concept should be the same.

marsnebulasoup · Accepted Answer · 2020-09-06 21:39:17Z

688

_.unique no longer works for the current version of Lodash as version 4.0.0 has this breaking change. The functionality of _.unique was splitted into _.uniq, _.sortedUniq, _.sortedUniqBy, and _.uniqBy.

You could use _.uniqBy like this:

_.uniqBy(data, function (e) {
  return e.id;
});

...or like this:

_.uniqBy(data, 'id');

Documentation: https://lodash.com/docs#uniqBy

For older versions of Lodash (< 4.0.0 ):

Assuming that the data should be uniqued by each object's id property and your data is stored in data variable, you can use the _.unique() function like this:

_.unique(data, function (e) {
  return e.id;
});

Or simply like this:

_.uniq(data, 'id');

edited Sep 6, 2020 at 21:39

marsnebulasoup

2,7003 gold badges20 silver badges40 bronze badges

answered Jul 31, 2015 at 7:17

ntalbs

29.6k8 gold badges72 silver badges86 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Will Lovett Over a year ago

The lodash 4.0.1 docs have changed this to uniqBy. Couldn't get the above example to work for the life of me using only uniq.

ersefuril Over a year ago

With lodash 4.x, uniq() accept only one parameter, which is an array. So as @WillLovett said, we can use uniqBy() to solve this problem.

Josh M. Over a year ago

Oh boy. Why, oh why, is it uniq which is pronounced you-knee-queue instead of what it should be, unique...

zdrsoft · Accepted Answer · 2016-11-18 11:33:27Z

95

You could use lodash method _.uniqWith, it is available in the current version of lodash 4.17.2.

Example:

var objects = [{ 'x': 1, 'y': 2 }, { 'x': 2, 'y': 1 }, { 'x': 1, 'y': 2 }];

_.uniqWith(objects, _.isEqual);
// => [{ 'x': 1, 'y': 2 }, { 'x': 2, 'y': 1 }]

More info: https://lodash.com/docs/#uniqWith

answered Nov 18, 2016 at 11:33

zdrsoft

2,69722 silver badges10 bronze badges

2 Comments

tagplus5 Over a year ago

` let uniqObjects = _.uniqBy(docs, function (e) { return e.x + e.y; }); ` This method will be faster for big arrays with simple objects

wsdfgfkdl Over a year ago

yes but you've to pass an attribute as first argument with uniqBy, which's not convenient if you want it unique based on several attributes that you don't already know

Taylor Hawkes · Accepted Answer · 2019-11-06 21:06:49Z

31

With lodash version 4+, you would remove duplicate objects either by specific property or by the entire object like so:

var users = [
  {id:1,name:'ted'},
  {id:1,name:'ted'},
  {id:1,name:'bob'},
  {id:3,name:'sara'}
];
var uniqueUsersByID = _.uniqBy(users,'id'); //removed if had duplicate id
var uniqueUsers = _.uniqWith(users, _.isEqual);//removed complete duplicates

Source:https://www.codegrepper.com/?search_term=Lodash+remove+duplicates+from+array

answered Nov 6, 2019 at 21:06

Taylor Hawkes

6617 silver badges10 bronze badges

2 Comments

Sydney C. Over a year ago

Was looking for that var uniqueUsers = _.uniqWith(users, _.isEqual);//removed complete duplicates Your anwsers deserves more love

Teebs Over a year ago

that's exactly what i was looking for _.uniqWith(users, _.isEqual) for unordered object

Vixson · Accepted Answer · 2020-04-22 01:26:28Z

24

Or simply Use union, for simple array.

_.union([1,2,3,3], [3,5])

// [1,2,3,5]

edited Apr 22, 2020 at 1:26

answered Mar 23, 2019 at 19:55

Vixson

7698 silver badges13 bronze badges

2 Comments

Sepehr Over a year ago

I like this answer. It's not the most readable one, but it actually works if you think about it from a mathematics perspective. _.union([1, 2, 2, 1, 3]) => [1, 2, 3]

Jean-Philippe Martin Over a year ago

clever and concise !!

ZettaP · Accepted Answer · 2020-06-09 12:09:04Z

22

For a simple array, you have the union approach, but you can also use :

_.uniq([2, 1, 2]);

answered Jun 9, 2020 at 12:09

ZettaP

1,49412 silver badges15 bronze badges

Comments

Hassan Ajaz · Accepted Answer · 2019-05-26 21:21:36Z

10

Simply use _.uniqBy(). It creates duplicate-free version of an array.

This is a new way and available from 4.0.0 version.

_.uniqBy(data, 'id');

or

_.uniqBy(data, obj => obj.id);

answered May 26, 2019 at 21:21

Hassan Ajaz

6738 silver badges16 bronze badges

Comments

Dmitry · Accepted Answer · 2021-11-04 18:15:37Z

In LODASH versions lower than 4 you will find most of this function are not implemented same way. And opposite from version 4 _.uniq was modified. I personally had a project that was in transition for a few months (from V3 -> to V4).

If you run in same situation and you have a lot of functions to be updated. You can do it in stages and when you are done with transition you can come and fix it later. This is the trick i used to avoid downtime of the platform:

/* LODASH Version 3 & 4 Compatibility Mode */
if ((_.VERSION).charAt(0) <= 3){ //Detect LODASH version 3 or 4.
    //V3 or lower
    _.uniq(data, true, 'id');
} else {
    //V4 or Higher
    _.uniqBy(data, 'id');
}

Also if you look at lodash documentation for most of this cases you can find migration of _.uniq from version lower than 4 can be performed with both functions:

_.uniqBy(data, 'id') or _.unionBy(data, 'id')

Both of them will bring same result. I personally was guessing which one to pick. In the end I picked this one: _.uniqBy(data, 'id').

Himanshu Tanwar · Accepted Answer · 2018-06-05 11:27:20Z

2

You can also use unionBy for 4.0.0 and later, as follows: let uniques = _.unionBy(data, 'id')

answered Jun 5, 2018 at 11:27

Himanshu Tanwar

3782 silver badges18 bronze badges

Collectives™ on Stack Overflow

Lodash remove duplicates from array

8 Answers 8

3 Comments

2 Comments

2 Comments

2 Comments

Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

8 Answers 8

3 Comments

2 Comments

2 Comments

2 Comments

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related