Remove duplicates from different JSON Arrays

Question

I have a JSON structure that has a lot of arrays.I want to check if there are duplicates.Not inside the same array but in different arrays.Also i want the other fields to stay as is.

Here is an example of my structure:

{
  "Collection":[
    {
    "field0":"string",
    "field1":"string",
    "field2":"string",
    "field3":"string",
    "field4":"string",
    "field5":"string",
    "field6":"string",
    "field7":"string",
    "field8":"string",
      "field9":[
         "test1"
         "test2"
         "test3"
       ]
    },
    {
    "field0":"string",
    "field1":"string",
    "field2":"string",
    "field3":"string",
    "field4":"string",
    "field5":"string",
    "field6":"string",
    "field7":"string",
    "field8":"string",
      "field9":[
         "test8"
         "test2"
         "test9"
       ]
    }
  ]
}

And here is what I expect:

{
  "Collection":[
    {
    "field0":"string",
    "field1":"string",
    "field2":"string",
    "field3":"string",
    "field4":"string",
    "field5":"string",
    "field6":"string",
    "field7":"string",
    "field8":"string",
      "field9":[
         "test1"
         "test2"
         "test3"
       ]
    },
    {
    "field0":"string",
    "field1":"string",
    "field2":"string",
    "field3":"string",
    "field4":"string",
    "field5":"string",
    "field6":"string",
    "field7":"string",
    "field8":"string",
      "field9":[
         "test8"
         "test9"
       ]
    }
  ]
}

I don't know if this is relevant but this is a firestore collection.

Hi there, Nick! Could you post what you have tried so far? So it helps understand your approach — natz
– natz, Commented Jun 12, 2019 at 13:18
What are you using to process this? Let us know which processing engine/programming language and anything you've tried so far! — Dan Gardner
– Dan Gardner, Commented Jun 12, 2019 at 13:18
Hello and you for your answers. This is a firestore collection and I am an IOS developer. So I am used to swift. I haven't tried anything because I think that I need python or something and I am not really used to. I need an opinion if I can achieve that. — Nick
– Nick, Commented Jun 12, 2019 at 13:28
Before you comment about iOS, I did working code for you in PHP. Sad :( — Serghei Niculaev
– Serghei Niculaev, Commented Jun 12, 2019 at 13:33

Serghei Niculaev · Accepted Answer · 2019-06-12 14:04:09Z

1

O'k, I don't know Swift, but I can show you how to do it:

<?php

$json = <<<JSON
{
    "Collection":[
        {
            "field1":[
                "test1",
                "test2",
                "test3"
            ]
        },
        {
            "field2":[
                "test8",
                "test2",
                "test9"
            ]
        }
    ]
}
JSON;

$entities = json_decode($json, true); // Decode JSON

$collection = $entities['Collection']; // Grab array of fields inside collection
$elements = []; // Initialize an empty array of unique elements
$result = [];
foreach ($collection as $index => $fieldObject) {
    $fieldName = array_keys($fieldObject)[0]; // Get field name

    // Get value from array of values of this field
    foreach ($fieldObject[$fieldName] as $valueKey => $value) {

        // Check if your value is not in array of unique elements
        if (!in_array($value, $elements)) {
            $elements[] = $value; // Add value if is not

            // Add value to your new array
            $result['Collection'][$index][$fieldName][] = $fieldObject[$fieldName][$valueKey];
        }
    }
}

$result = json_encode($result); // Encode it back to JSON

Here is the working example in sandbox: http://sandbox.onlinephpfunctions.com/code/a78cff49cc31c15e7e1373f7e1f66b7951f129e9

answered Jun 12, 2019 at 14:04

Serghei Niculaev

5157 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Nick Over a year ago

Thank you so much. How can i run it and make a new json without duplicates?

Serghei Niculaev Over a year ago

Hmm... Edit this algorithm using Swift :D Think of it as about pseudocode

Nick Over a year ago

One more thing. If i have more fields the code is the same?? because i have 1 field with array and 9 more fields with strings(not arrays).But i dont want to check for dublicates in the other fields only in the arrays.

Serghei Niculaev Over a year ago

You can check that fields are instance of array type iswift.org/cookbook/check-if-object-is-an-array and do this algorytm only for them

Collectives™ on Stack Overflow

Remove duplicates from different JSON Arrays

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related