Remove duplicates from NESTED array of objects

Question

I have an array of objects like below, which inside of it has another array of objects which has duplicate verticalName values that need to be removed in order to be displayed on the page.

 "landingPages": [
        {
          "programmes": [
            {
              "progName": "Programme 1",
              "verticals": [
                {
                  "id": "8",
                  "verticalName": "Law and Criminology"
                }
              ]
            }
          ]
        },
        {
          "programmes": [
            {
              "progName": "Programme 2",
              "verticals": [
                {
                  "id": "1",
                  "verticalName": "Psychology and Sociology"
                }
              ]
            }
          ]
        },
        {
          "programmes": [
            {
              "progName": "Programme 3",
              "verticals": [
                {
                  "id": "3",
                  "verticalName": "Computing and IT"
                }
              ]
            }
          ]
        },
        {
          "programmes": [
            {
              "progName": "Programme 4",
              "verticals": [
                {
                  "id": "1",
                  "verticalName": "Psychology and Sociology"
                }
              ]
            },
            {
              "progName": "Programme 5",
              "verticals": [
                {
                  "id": "3",
                  "verticalName": "Computing and IT"
                }
              ]
            },
            {
              "progName": "Programme 6",
              "verticals": [
                {
                  "id": "2",
                  "verticalName": "Business and Management"
                }
              ]
            },
            {
              "progName": "Programme 7",
              "verticals": [
                {
                  "id": "3",
                  "verticalName": "Computing and IT"
                }
              ]
            },
            {
              "progName": "Programme 8",
              "verticals": [
                {
                  "id": "3",
                  "verticalName": "Computing and IT"
                }
              ]
            }
          ]
        }
      ]

I have tried couple of solutions, some of the did not work but one did, which I copy/pasted below. I've managed to store all of them inside of one array of objects, it's just that the code to me looks really bad. I am trying to find a cleaner solution to this problem. Here is my solution and I would like to see how does it compare to your solutions maybe, and if you could give me some feedback that would be amazing.

    let known = {}
    let noduplicates = programmes.map((subarray) => {
        const newarrays = []
        subarray.verticals.forEach((item) => {
            if (
                !known.hasOwnProperty(item.verticalName) &&
                (known[item.verticalName] = true)
            ) {
                newararys.push(item)
            } else {
                console.log('test')
            }
        })
        console.log(newarrays)
        return newarrays
    })
    const fil = filtered.filter((item) => item.length)
    const singleArr = fil.reduce((a, b) => a.concat(b), [])
    console.log(singleArr)

What I am trying to get is something like this, basically, just all the objects that were found inside of verticals array, but - without duplicates:

[
   {id: "1", verticalName: "Psychology and Sociology"}, 
   {id: "3", verticalName: "Computing and IT"}, 
   {id: "2", verticalName: "Business and Management"}
]

Cheers everyone!

define duplicate. verticals? then which progName stay? or does progName does not matter? How you want your final array to look like? — Someone Special
– Someone Special, Commented May 11, 2021 at 13:04
Hi, I updated the question. It's the verticalName inside key inside of the nested "vertical" array. It doesn't matter, I just need to store all the non duplicate values inside of one array of objects. — Marko
– Marko, Commented May 11, 2021 at 13:07

dbuchet · Accepted Answer · 2021-05-11 13:34:38Z

2

You could achieve it by pure ES6 functions, I'll decompose, but you can have it all in one line :)

first, you need to retrieve all verticalNames

const allVerticalNames = landingPages.map(lp => lp.programmes.map(p => p.verticals))

You'll have plenty arrays or arrays, so we need to flatten all of this

const flattened = allVerticalNames.flat(2)

At this point you have all your verticalNames, but not uniq. To do that, we should convert it to a Set, Map or easier, an Object

const keyValuePairs = flattened.map(v => [v.id, v.verticalName]); // We transform to a pair key / value
const uniq = Object.fromEntries(keyValuePairs)

At this point you have an object like {1: "Psychology and Sociology", 2: "Business and Management", 3: "Computing and IT", 8: "Law and Criminology"} So if you want to have it back to the way in exemple, you move it back to an array!

const final = Object.entries(uniq).map(([id, verticalName]) => ({id, verticalName}))

And that's it!

All in one line?

const final = Object.entries(Object.fromEntries(landingPages.map(lp => lp.programmes.map(p => p.verticals)).flat(2).map(v => [v.id, v.verticalName]))).map(([id, verticalName]) => ({id, verticalName}))

answered May 11, 2021 at 13:34

dbuchet

1,6511 gold badge7 silver badges7 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

painotpi Over a year ago

Based on the expected outcome in the question, I think OP just wants duplicates eliminated from the nested array, the unique values don't necessarily depend on the values of verticals at the root level of the landingPages array. Your answer is great tbh, but the output includes Law and Criminology which is not in the expected outcome.

dbuchet Over a year ago

I tought we needed all the uniq values across all programs. We can adapt the logic very easily to remove duplicate in each program only thought ;)

Marko Over a year ago

Cleanest solution to be honest, and a great explanation of each step

Aaron Meese · Accepted Answer · 2022-05-26 09:40:16Z

You may use set for this use case which lets you store unique values of any type.

const data = [
        {
          "programmes": [
            {
              "progName": "Programme 1",
              "verticals": [
                {
                  "id": "8",
                  "verticalName": "Law and Criminology"
                }
              ]
            }
          ]
        },
        {
          "programmes": [
            {
              "progName": "Programme 2",
              "verticals": [
                {
                  "id": "1",
                  "verticalName": "Psychology and Sociology"
                }
              ]
            }
          ]
        },
        {
          "programmes": [
            {
              "progName": "Programme 3",
              "verticals": [
                {
                  "id": "3",
                  "verticalName": "Computing and IT"
                }
              ]
            }
          ]
        },
        {
          "programmes": [
            {
              "progName": "Programme 4",
              "verticals": [
                {
                  "id": "1",
                  "verticalName": "Psychology and Sociology"
                }
              ]
            },
            {
              "progName": "Programme 5",
              "verticals": [
                {
                  "id": "3",
                  "verticalName": "Computing and IT"
                }
              ]
            },
            {
              "progName": "Programme 6",
              "verticals": [
                {
                  "id": "2",
                  "verticalName": "Business and Management"
                }
              ]
            },
            {
              "progName": "Programme 7",
              "verticals": [
                {
                  "id": "3",
                  "verticalName": "Computing and IT"
                }
              ]
            },
            {
              "progName": "Programme 8",
              "verticals": [
                {
                  "id": "3",
                  "verticalName": "Computing and IT"
                }
              ]
            }
          ]
        }
      ];
      
      const seen = new Set();
      const filteredArr = data.filter(el => {
        const verticalName = el.programmes[0].verticals[0].verticalName;
        const duplicate = seen.has(verticalName);
        seen.add(verticalName);
        return !duplicate;
      });

      console.log(filteredArr);

painotpi · Accepted Answer · 2021-05-11 14:05:26Z

If I understand the question correctly, we need to remove the verticalNames from the nested programmes array.

const landingPages = [
  {
    programmes: [
      {
        progName: 'Programme 1',
        verticals: [
          {
            id: '8',
            verticalName: 'Law and Criminology',
          },
        ],
      },
    ],
  },
  {
    programmes: [
      {
        progName: 'Programme 2',
        verticals: [
          {
            id: '1',
            verticalName: 'Psychology and Sociology',
          },
        ],
      },
    ],
  },
  {
    programmes: [
      {
        progName: 'Programme 3',
        verticals: [
          {
            id: '3',
            verticalName: 'Computing and IT',
          },
        ],
      },
    ],
  },
  {
    programmes: [
      {
        progName: 'Programme 4',
        verticals: [
          {
            id: '1',
            verticalName: 'Psychology and Sociology',
          },
        ],
      },
      {
        progName: 'Programme 5',
        verticals: [
          {
            id: '3',
            verticalName: 'Computing and IT',
          },
        ],
      },
      {
        progName: 'Programme 6',
        verticals: [
          {
            id: '2',
            verticalName: 'Business and Management',
          },
        ],
      },
      {
        progName: 'Programme 7',
        verticals: [
          {
            id: '3',
            verticalName: 'Computing and IT',
          },
        ],
      },
      {
        progName: 'Programme 8',
        verticals: [
          {
            id: '3',
            verticalName: 'Computing and IT',
          },
        ],
      },
    ],
  },
]

let page = landingPages.find(page => page.programmes.length > 1);
const uniqueprogrammes = new Set();

const uniqueverticals = page.programmes.map(p => {
  const vn = p.verticals[0].verticalName;
  if (!uniqueprogrammes.has(vn)) {
    uniqueprogrammes.add(vn);
    return p.verticals[0];
  }
}).filter(v => !!v);

console.log(uniqueverticals);

UPDATE:

Updated code snippet based on the expected outcome in the question.

The idea is to first find the nested array of programmes which is done by filtering the landingPages object based on the .length

let page = landingPages.find(page => page.programmes.length > 1);

Then, we create a set to keep a track of all the unique programmes from this nested array,

const uniqueprogrammes = new Set();

Next, we map through the nested array and find all the verticals that are not a part of the uniqueprogrammes set,

const uniqueverticals = page.programmes.map(p => {
  const vn = p.verticals[0].verticalName;
  if (!uniqueprogrammes.has(vn)) {
    uniqueprogrammes.add(vn);
    return p.verticals[0];
  }
})

However, this will gives us an output like the following,

[
  { id: '1', verticalName: 'Psychology and Sociology' },
  { id: '3', verticalName: 'Computing and IT' },
  { id: '2', verticalName: 'Business and Management' },
  undefined,
  undefined
]

so (lastly), we need to filter out the falsy values with,

.filter(v => !!v);

Ankit · Accepted Answer · 2021-05-11 13:14:58Z

-1

You can use loadash to flatter and then remove duplicates.

var arrayWithDuplicatesRemoved=.flattenDeep(landingPages);

arrayWithDuplicatesRemoved = arrayWithDuplicatesRemoved.uniqBy(data, function (e) {return e.verticalName;});

answered May 11, 2021 at 13:14

Ankit

452 bronze badges

2 Comments

Brain Foo Long Over a year ago

Suggesting external libs for basic JS functionality is not that ideal.

Mulan Over a year ago

@BrainFooLong the question is tagged with lodash

Collectives™ on Stack Overflow

Remove duplicates from NESTED array of objects

4 Answers 4

3 Comments

Comments

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

3 Comments

Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related