Numpy: Loading data from JSON file gives np array with nested lists

Question

I am loading data from a JSON file as list and then making a NumPy array.

The JSON file is structured as follows

{
    "label": "4",
    "mfcc": [
        [
            [
                -147.2358550730904,
                52.60503152410914,
                <more values Total=13>

            ],
            <more arrays Total=44>

The code I am using to make a NumPy array using data collected from the JSON file

with open("data.json", 'r') as file:
   data = json.load(file)
   
   mfcc = np.array(data["mfcc"])

It seems that the most outer list gets converted to a NumPy array while the inner lists are still lists. See the image below:

What has happened?

Thanks in advance!

A JSON string/file can be a combination of nested lists and dicts. You don't show any of file or the list after loading. np.array(alist) will give a nice multdimensional array only if the nesting in the list is regular (as shown in numpy.array docs. So just saying that the file is JSON doesn't help us help you. — hpaulj
– hpaulj, Commented Oct 14, 2020 at 20:00
data["mfcc"] does look like a list, but it may not be regular enough to turn into a multidimensional array. — hpaulj
– hpaulj, Commented Oct 15, 2020 at 22:45
The list was irregular. I added a condition in the code that made the JSON file to deal with it. Thank you. — InputBlackBoxOutput
– InputBlackBoxOutput, Commented Oct 17, 2020 at 13:35

Valentin Vignal · Accepted Answer · 2020-10-14 14:39:03Z

1

So it means the file contains a numpy array with lists inside.

Either is was made on purpose
Either you (or someone) tried to convert a list of list into a numpy array but the nested lists are not all with the same length (which is required for a numpy array). Therefore, numpy doesn't create a proper 2D numpy array but a 1D numpy array with lists inside. To fix it, make sure all the nested lists are the same length (you can pad them with 0 for instance)

answered Oct 14, 2020 at 14:39

Valentin Vignal

8,6444 gold badges47 silver badges99 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

InputBlackBoxOutput Over a year ago

The second point does not hold.The data in the JSON file is complete

InputBlackBoxOutput Over a year ago

Rechecked the JSON file. Missing values was the imposter. Thanks

Valentin Vignal Over a year ago

Glad you found the issue. Please don't forget to validate the answer or add the answer to your question for the future readers

Collectives™ on Stack Overflow

Numpy: Loading data from JSON file gives np array with nested lists

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related