Filter an array to remove duplicates in JS

Question

I've scoured through many posts and have been working on this problem for too long. The gist is that I want to filter an array if there are duplicate object id's and return the object with the lower value or "val", otherwise return the original element.

Start with:

**edited input for clarification

let input = [
    {"id": 1, "val": 3},
    {"id": 2, "val": 1},
    {"id": 3, "val": 4},
    {"id": 1, "val": 0}
]

function removeDuplicates(array, propertyName) {
    // where propertyName is "id"
}

The result of the function should be:

[
    {"id": 1, "val": 0},
    {"id": 2, "val": 1},
    {"id": 3, "val": 4}
]

The naive implementation would be exactly as if you were doing it on a piece of paper. Would you be able that manually if you are given a moderate size list (like 50 elements) with pen and paper? — zerkms
– zerkms, Commented Nov 12, 2017 at 22:55
@Salman not necessarily, and you only use "val" when you have to objects that have the same "id". When comparing, the object with the lower "val" would be returned in the array. — simplyketchup
– simplyketchup, Commented Nov 12, 2017 at 22:55
@Tyler Not sure if I understood your question, but does my snippet work for you or should it be the other way round? — Praveen Kumar Purushothaman
– Praveen Kumar Purushothaman, Commented Nov 12, 2017 at 22:59
I would suggest use Lodash uniqBy Which you can find it here. lodash.com/docs/4.17.4#uniqBy — bvmCoder
– bvmCoder, Commented Nov 12, 2017 at 23:06

charlietfl · Accepted Answer · 2017-11-12 23:15:39Z

3

A very common approach to such issues is to create a temporary object that uses the common property as keys and a full object as values.

Once you loop through the data array once and build the object then it is a simple process to put the values into resultant array

let input = [
    {"id": 1, "val": 0},
    {"id": 2, "val": 0},
    {"id": 3, "val": 0},
    {"id": 1, "val": 1}
]

let tmp = {}

input.forEach(o=> {
  tmp[o.id] = tmp[o.id] || o;
  // assign lower value if applicable
  if( o.val < tmp[o.id].val){         
    tmp[o.id].val = o.val;
  }
});

let res = Object.values(tmp);

console.log(res);

answered Nov 12, 2017 at 23:15

charlietfl

172k14 gold badges125 silver badges154 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

JohanP · Accepted Answer · 2017-11-12 23:21:43Z

0

You can use reduce and then check in the accumulator if current value is smaller than one you have already saved.

let input = [
    {"id": 1, "val": 0},
    {"id": 2, "val": 0},
    {"id": 3, "val": 0},
    {"id": 1, "val": 1}
]

function removeDuplicates(array, propertyName) {
    return array.reduce((acc, cv) => {
        if (acc.hasOwnProperty(cv[propertyName])) {
            if (cv.val < acc[cv[propertyName]].val)
                acc[cv[propertyName]] = cv;
        } else
            acc[cv[propertyName]] = cv;
        return acc;
    }, {});
}
console.log(removeDuplicates(input,"id"))

answered Nov 12, 2017 at 23:21

JohanP

5,5423 gold badges27 silver badges37 bronze badges

Comments

AskYous · Accepted Answer · 2017-11-12 23:21:49Z

0

Use .filter and .map:

inputs
    .filter(object => object.id == propertyValue) // get only the objects you want
    .map(object => object.val)                    // convert each object to it's value
    .sort()                                       // sort it
    [0]                                           // and get the first (meaning smallest) element

answered Nov 12, 2017 at 23:21

AskYous

4,8089 gold badges57 silver badges92 bronze badges

7 Comments

Ben Hull Over a year ago

This looks very neat, but I'm not sure how it works! Can you explain what's going on in the object.id == propertyValue bit? It doesn't run for me here: jsfiddle.net/12bozrnv/1

AskYous Over a year ago

@Beejamin, the jsfiddle doesn't run because propertyValue is undefined. But in that line of code I wrote, I used the filter function with arrow functions. The filter function takes a function that returns true if the object satisfied the filter algorithm. Here's an example where I filter for numbers only: myArray.filter(x => typeof x == 'number'). This returns only numbers from the myArray array.

Ben Hull Over a year ago

propertyValue is a placeholder? Gotcha.

AskYous Over a year ago

@Beejamin It was a reference to the OP's code. His function is given propertyName. Though I accidentally used propertyValue.

Ben Hull Over a year ago

I've tried to adjust this to make a runnable example, but I can't get it to work. I'm not super familiar with ECMA6 conventions though, so it could be that. Would you mind expanding your example to a runnable snippet?

|

Ori Shalom · Accepted Answer · 2017-11-12 23:30:41Z

0

The most efficient approach is to simply use a temp object to store all the objects in the array with the ids as object keys, since object key can't be duplicate, multiple occurrence will simply override each other till finally you will just have to convert the object back into an array using Object.values(obj).

var input = [
    {"id": 1, "val": 0},
    {"id": 2, "val": 0},
    {"id": 3, "val": 0},
    {"id": 1, "val": 1}
];
function removeDuplicates(array, propertyName) {
    var tmp = {};
    array.forEach(function (v) {
        if (tmp[v[propertyName]] == null) {
            tmp[v[propertyName]] = v;
        } else if (v.val < tmp[v[propertyName]].val)
            tmp[v[propertyName]] = v;
        }
    });
    return Object.values(tmp);
}

Use this:

removeDuplicates(input, "id");

To return the following:

[
    {"id": 1, "val": 0},
    {"id": 2, "val": 0},
    {"id": 3, "val": 0}
]

edited Nov 12, 2017 at 23:30

answered Nov 12, 2017 at 23:07

Ori Shalom

5004 silver badges11 bronze badges

1 Comment

simplyketchup Over a year ago

Testing the code, it's returning the { "id": 1, "val": 1 }. Switching the "val"s around, it looks like this code produces the second { "id": 1 } every time

Ben Hull · Accepted Answer · 2017-11-13 05:48:16Z

0

You can use Array.reduce to build a new hash out of the lowest values, then just return the values of that hash.

var input = [
  {"id": 1, "val": 0},
  {"id": 2, "val": 0},
  {"id": 3, "val": 0},
  {"id": 1, "val": 1}
];

function minFromArray(ar) {
  //Object.values returns the values of a hash as an array.
  return Object.values(ar.reduce(function(acc, obj){
    //If this is the first instance of the ID, or the value is lower than the existing
    if (typeof(acc[obj['id']]) == 'undefined' || obj['val'] < acc[obj['id']]) {
      acc[obj['id']] = obj;
    }
    return acc;
  }, {}));
}

console.log(minFromArray(input));

edited Nov 13, 2017 at 5:48

answered Nov 12, 2017 at 23:27

Ben Hull

7,6653 gold badges40 silver badges58 bronze badges

Collectives™ on Stack Overflow

Filter an array to remove duplicates in JS

5 Answers 5

Comments

Comments

7 Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

Comments

7 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related