Javascript - clearing duplicates from an array object

Question

Hi I have a javascript array object rapresenting the amount of items sold in a given country, like this:

var data = [{'c1':'USA', 'c2':'Item1', 'c3':100}, 
            {'c1':'Canada', 'c2':'Item1', 'c3':120},
            {'c1':'Italy', 'c2':'Item2', 'c3':140},
            {'c1':'Italy', 'c2':'Item2', 'c3':110}]

I need to avoid duplicates (as you may see, the last two 'records' have the same Country and the same Item) and sum the amounts; if I was getting data from a database I would use the DISTINCT SUM clause, but what about it in this scenario? Is there any good jquery trick?

Hm. The last two records do not really have the same item in your sample. — Tomalak
– Tomalak, Commented Dec 27, 2010 at 13:01
You could try to construct a new data structure (Object) (key = country name, value = rest of data). Check if key exists in the structure already while constructing the new one and sum accordingly. There might be a neater way... — Juho Vepsäläinen
– Juho Vepsäläinen, Commented Dec 27, 2010 at 13:03

T.J. Crowder · Accepted Answer · 2010-12-27 13:33:01Z

6

You could use an object as a map of distinct values, like this:

var distincts, index, sum, entry, key;
distincts = {};
sum = 0;
for (index = 0; index < data.length; ++index) {
    entry = data[index];
    key = entry.c1 + "--sep--" + entry.c2;
    if (!distincts[key]) {
        distincts[key] = true;
        sum += entry.c3;
    }
}

How that works: JavaScript objects are maps, and since access to properties is an extremely common operation, a decent JavaScript implementation tries to make property access quite fast (by using hashing on property keys, that sort of thing). You can access object properties using a string for their name, by using brackets ([]), so obj.foo and obj["foo"] both refer to the foo property of obj.

And so:

We start with an object with no properties.
As we loop through the array, we create unique key from c1 and c2. It's important that the "--sep--" string be something that cannot appear in c1 or c2. If case isn't significant, you might throw a .toLowerCase in there.
If distincts already has a value for that key, we know we've seen it before and we can ignore it; otherwise, we add a value (true in this case, but it can be just about anything other than false, undefined, 0, or "") as a flag indicating we've seen this unique combination before. And we add c3 to the sum.

But as someone pointed out, your last two entries aren't actually the same; I'm guessing that was just a typo in the question...

edited Dec 27, 2010 at 13:33

answered Dec 27, 2010 at 13:03

T.J. Crowder

1.1m201 gold badges2k silver badges2k bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

T.J. Crowder Over a year ago

@Tomalek: Took typing in high school. ;-)

franz976 Over a year ago

It works. To be honest, I was expecting something a little less tricky than concatenating strings, but it's probably because of my ignorance about javascript. Thank you TJ.

T.J. Crowder Over a year ago

@franz: Well, you could do a two-level deep thing, with the first level keyed by c1 and the second level keyed by c2, but frankly it would make the code more complicated and wouldn't buy you anything at runtime. :-)

mwilcox · Accepted Answer · 2010-12-27 13:05:45Z

0

jQuery may have an array function for this, but because your two Italy objects are not distinctly unique, your asking for a custom solution. You want to populate a array and check it for duplicates as you go:

var data = [{'c1':'USA', 'c2':'Item1', 'c3':100}, 
            {'c1':'Canada', 'c2':'Item1', 'c3':120},
            {'c1':'Italy', 'c2':'Item2', 'c3':140},
            {'c1':'Italy', 'c2':'Item1', 'c3':110}]

var newArray = [];
var dupeCheck = {}; // hash map
for(var i=0; i < data.length; i++){
   if(!dupeCheck[data[i].c1]){
      newArray.push(data[i]);
      dupeCheck[data[i].c1] = true;
   }
}

answered Dec 27, 2010 at 13:05

mwilcox

4,1321 gold badge26 silver badges20 bronze badges

Comments

andres descalzo · Accepted Answer · 2010-12-27 19:41:27Z

-1

test

HTML:

<div id="test"></div>

JS:

var data = [{'c1':'USA', 'c2':'Item1', 'c3':100}, 
            {'c1':'Canada', 'c2':'Item1', 'c3':120},
            {'c1':'Italy', 'c2':'Item2', 'c3':140},
            {'c1':'Italy', 'c2':'Item1', 'c3':110}];

var
 l = data.length, // length
 f = "",          // find
 ix = "",         // index
 d = [];          // delete    

for (var i = 0; i < l; i++) {

   ix = data[i].c1 + "_" + data[i].c2 + "__";

   //var re = new RegExp(ix);

   //if (re.test(f))
   if (f.indexOf(ix) != -1)
     d.push(i);
   else
     f += ix;
}


for (var i1 = 0; i1 < d.length; i1++){

    $("#test").append("<div>for delete: "+d[i1]+"</div>");

}

EDIT

Although chrome works much faster, works only in chrome faster the example with indexOf, then in IE/Opera/Firefox/Safary works faster with an object.

The code created by "@ TJ Crowder" is much more efficient.

edited Dec 27, 2010 at 19:41

answered Dec 27, 2010 at 13:56

andres descalzo

15k14 gold badges66 silver badges119 bronze badges

1 Comment

T.J. Crowder Over a year ago

Er...and if the values in c1 or c2 have characters in them that are special in RegExps? (Think *, for instance.) Or _ s? And...why the regexp, exactly? This is basically: Create one long string with keys (you don't need regexp for that) and then look to see if a key is already in the string.

Collectives™ on Stack Overflow

Javascript - clearing duplicates from an array object

3 Answers 3

3 Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related