how to remove duplicate objects from an Array of objects but with two unique keys [duplicate]

Question

So I have an array of objects and I am trying to remove the duplicates. For example, lets say my array of object contains this data:

[{TERM_CD: "220181", CRS_SUBJ_CD: "LALS", CRS_SUBJ_DESC: "Latin American &Latino Studies", CRS_NBR: "127", CRS_TITLE: "Latin American Music", …}, 
{TERM_CD: "220101", CRS_SUBJ_CD: "MUS", CRS_SUBJ_DESC: "Music", CRS_NBR: "127", CRS_TITLE: "Latin American Music", …}, etc...]

And this is the removeDup function that I am using to remove duplicates:

removeDup(data, key) {
    return [
      ...new Map(
        data.map(x=>[key(x), x])
      ).values(),
    ];
  }

For example if I call the remove dup function like this then one of the object will be removed since they both have same crs_title.

const noDupArr = this.removeDup(printArr, x => x.CRS_TITLE);

So my goal is to try to remove duplicate on 2 keys CRS_SUBJ_CD and CRS_NBR but I am not able to figure out how to proceed doing that. I tried adding another key parameter in the removeDup function but wasn't successful. Any ideas on fixing this issue. Thank you in advance!

my goal is to try to remove duplicate on 2 keys CRS_SUBJ_CD and CRS_NBR. If your removeDup function is working, then why not call it twice with the other keys you want to remove — smac89
– smac89, Commented Jun 26, 2020 at 18:22
Calling it twice isn't really solving the issue because it removes certain objects that shouldn't be removed. — Shalin Patel
– Shalin Patel, Commented Jun 26, 2020 at 18:24
You can think of using Map , where title is key and full obj is value. If it is already present , dont add else add. — Harmandeep Singh Kalsi
– Harmandeep Singh Kalsi, Commented Jun 26, 2020 at 18:25
@Henrique Viana that works! Didn't get to look at the question before. Thanks! — Shalin Patel
– Shalin Patel, Commented Jun 26, 2020 at 18:41

Rajneesh · Accepted Answer · 2020-06-26 18:39:31Z

1

You can maintain an array of keys for filtering and then make use of Map to get unique values. Here is an example with duplicate data:

var data=[{TERM_CD: "220181", CRS_SUBJ_CD: "LALS", CRS_SUBJ_DESC: "Latin American &Latino Studies", CRS_NBR: "127", CRS_TITLE: "Latin American Music"},
{TERM_CD: "220101", CRS_SUBJ_CD: "MUS", CRS_SUBJ_DESC: "Music", CRS_NBR: "127", CRS_TITLE: "Latin American Music"},{TERM_CD: "220101", CRS_SUBJ_CD: "MUS", CRS_SUBJ_DESC: "Music", CRS_NBR: "127", CRS_TITLE: "Latin American Music"}];

var keys = ['CRS_SUBJ_CD',  'CRS_NBR'];

var result = [...new Map(data.map(p=>[keys.map(k=>p[k]).join('|'),p])).values()];

console.log(result);

I hope this helps. Thanks!

answered Jun 26, 2020 at 18:39

Rajneesh

5,3181 gold badge10 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

smac89 Over a year ago

False positives: {CRS_SUBJ_CD:"foo", CRS_NBR:"bar"} and {CRS_SUBJ_CD:"foo|bar", CRS_NBR:""} OR {CRS_SUBJ_CD:"foo ", CRS_NBR:"|bar"} and {CRS_SUBJ_CD:"foo |", CRS_NBR:"bar"}

Collectives™ on Stack Overflow

how to remove duplicate objects from an Array of objects but with two unique keys [duplicate]

1 Answer 1

1 Comment

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Linked

Related