Fast lookup of integer index value using a sorted integer array key

Question

I'm looking for the fastest solution to looking up an integer Value using a sorted integer array Key.

The Keys are integer arrays and have a fixed length of 3 and each array is sorted.
The Value is an integer.

My data guarantees that there are only EVER one OR two sorted arrays that have the same content. Each array has a unique index.

I'm trying to find the matching pairs of arrays.

My thought is to use a dictionary (I'm prototyping in C# and will move to C++)

For each array, I'll look in the dictionary and see if it's already there. If it is, I remove it from the dictionary. If I don't find it in the dictionary then it's either a singleton or it's the first of a matching pair so I'll add it to the dictionary.

My question is this - give then very specific guarantees on the data, what's the best container - given that speed is my primary concern? Also, any recommendations on appropriate (fast) hashing functions or compare functions for sorted integer arrays would be appreciated.

if your code ultimately need to be C++, don't waste time over C#. what you found working fast in C#, does not mean you get the same result in C++. — Kelmen
– Kelmen, Commented Apr 12, 2013 at 3:57
CRC32 is a fast hashing function with no cryptographic guarantees. Also, unless you expect to get a lot of arrays having the same first X entries, you could hash on only the first X entries to save time. — Patashu
– Patashu, Commented Apr 12, 2013 at 4:09

Steve Kallestad · Accepted Answer · 2013-04-12 04:12:42Z

2

When you get to C++, migrate to this

http://sparsehash.googlecode.com/svn/trunk/doc/dense_hash_map.html (Project here.)

It's one of the fastest hashmap implementations that I've run into.

In the mean time, for C# the equivalent would be something like this: http://msdn.microsoft.com/en-us/library/xfhwa508.aspx I imagine there are faster dictionary implementations available, but since C# is not the final container, it should do just fine.

You might want to think about including berkeleydb in your project. It's VERY fast and manages storage as well for when the dataset grows. It's also supported on a wide variety of platforms.

answered Apr 12, 2013 at 4:12

Steve Kallestad

3,5933 gold badges26 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Fast lookup of integer index value using a sorted integer array key

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related