Passing multidimensional arrays from a perl script to a python script

Question

here is what I am doing:

I am taking a multidimensional array, or rather, an array of hashes and trying to pass it into a python script from my perl script. Currently I am converting it to json, and then passing the json string as a literal string into the python script as a parameter.

The array of hashes looks like the following example:

 %HoH = (
id10001 => {
    lat   => "180",
    long  => "-180",
},
id10002 => {
    lat   => "180",
    long  => "-180",
},
id10003 => {
    lat   => "180",
    long  => "-180",
}
);

which i then inside of my perl script, turn into a json string:

{
    "id10001": { "lat": "180", "lon": "-180" },
    "id10002": { "lat": "180", "lon": "-180" },
    "id10003": { "lat": "180", "lon": "-180" },
}

which then is passed into a python script. The python script decodes the json string back into the original constructed array structure.

Is there a better way to pass arrays, or multidimensional arrays from a perl script to a python scrip?

thank you in advance for your help

JSON is a pretty fast format when serializing, so in that regard it's pretty fast. Depending on which library you're using in Python, you might be able to get a significant performance boost by switching to one written in C. simpleJSON is much faster than the standard json module. — Morten Jensen
– Morten Jensen, Commented Jul 3, 2012 at 16:13
Not sure how your two scripts are interacting but you could also set something up with shared memory and pack and unpack the structure binary data directly. It certainly would require some more work but it probably would be faster — Paul Seeb
– Paul Seeb, Commented Jul 3, 2012 at 16:24
Why do you have the extra layer of list wrapping in the serialized form? — Karl Knechtel
– Karl Knechtel, Commented Jul 3, 2012 at 17:14
Thanks for the responses. I mistype my json when i made the example. — Beau Bouchard
– Beau Bouchard, Commented Jul 5, 2012 at 15:06

Oleg V. Volkov · Accepted Answer · 2012-07-03 16:42:37Z

7

No, serialization is the only way to pass data through plain character buffer (such as command line arguments, file or whatever) by its very definition. As long as specific serialization format - JSON is this case - naturally covers language-specific structures, you're all set.

If you so wish you can experiment with benchmarking libraries for serialization formats available to both Perl and Python to see which will be faster in your case, but at least Perl's XS implementation of JSON is known to be very fast and routinely beat in speed other available serializers.

edited Jul 3, 2012 at 16:42

answered Jul 3, 2012 at 16:14

Oleg V. Volkov

22.6k4 gold badges49 silver badges68 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Karl Knechtel · Accepted Answer · 2012-07-03 17:17:40Z

1

Your JSON seems oddly structured; the analogous structure to what you've shown in Perl would be more like

{
    "id10001": { "lat": "180", "lon": "-180" },
    "id10002": { "lat": "180", "lon": "-180" },
    "id10003": { "lat": "180", "lon": "-180" },
}

(Despite your bracketing conventions in Perl, you have a hash of hashes, not an array of hashes. Also, is there any particular reason for using strings rather than numbers for the lat/lon values?)

But other than that, you have the right idea.

answered Jul 3, 2012 at 17:17

Karl Knechtel

61.4k14 gold badges131 silver badges193 bronze badges

1 Comment

Beau Bouchard Over a year ago

Yes, The information I am using are user generated locations. I could not directly copy and paste so I made a generic example. The lat longs are only strings because I rendered them so when I made my fictional example of data. Thank you all for your excellent input

Collectives™ on Stack Overflow

Passing multidimensional arrays from a perl script to a python script

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related