How do I efficiently write a Python dictionary, where the values are Numpy Nd-Arrays to a Json File? I obtain an error saying that the Numpy Nd-Array is not Json-Serializable. Is there any way to overcome this?
2
-
Did you try converting the Numpy arrays to (possibly nested) lists?PM 2Ring– PM 2Ring2017-06-01 08:34:15 +00:00Commented Jun 1, 2017 at 8:34
-
yes, it is possible to convert the Numpy arrays to lists and it then works. My question goes rather in other direction: is there any quicker way to achieve this purpose, without all the converting and copying to lists?AlexGuevara– AlexGuevara2017-06-01 08:46:08 +00:00Commented Jun 1, 2017 at 8:46
Add a comment
|
2 Answers
JSON only supports a limited number of datatypes. If you want to store other types of data as JSON then you need to convert it to something that JSON accepts. The obvious choice for Numpy arrays is to store them as (possibly nested) lists. Fortunately, Numpy arrays have a .tolist method which performs the conversion efficiently.
import numpy as np
import json
a = np.array(range(25), dtype=np.uint8).reshape(5, 5)
print(a)
print(json.dumps(a.tolist()))
output
[[ 0 1 2 3 4]
[ 5 6 7 8 9]
[10 11 12 13 14]
[15 16 17 18 19]
[20 21 22 23 24]]
[[0, 1, 2, 3, 4], [5, 6, 7, 8, 9], [10, 11, 12, 13, 14], [15, 16, 17, 18, 19], [20, 21, 22, 23, 24]]
.tolist will convert the array elements to native Python types (int or float) if it can do so losslessly. If you use other datatypes I suggest you convert them to something portable before calling .tolist.
1 Comment
JoAnn Alvarez
OP wants to serialize a dictionary of arrays.
Here is a full working example of an Encoder/Decoder that can deal with NumPy arrays:
import numpy
from json import JSONEncoder,JSONDecoder
import json
# ********************************** #
class NumpyArrayEncoder(JSONEncoder):
def default(self, obj):
if isinstance(obj, numpy.ndarray):
return obj.tolist()
return JSONEncoder.default(self, obj)
class NumpyArrayDecoder(JSONDecoder):
def default(self, obj):
if isinstance(obj, list):
return numpy.asarray(obj)
return JSONEncoder.default(self, obj)
# ********************************** #
if __name__ == "__main__":
# TO TEST
numpyArrayOne = numpy.array([[11 ,22, 33], [44, 55, 66], [77, 88, 99]])
numpyArrayTwo = numpy.array([[51, 61, 91], [121 ,118, 127]])
# Serialization
numpyData = {"arrayOne": numpyArrayOne, "arrayTwo": numpyArrayTwo}
print("Original Data: \n")
print(numpyData)
print("\nSerialize NumPy array into JSON and write into a file")
with open("numpyData.json", "w") as write_file:
json.dump(numpyData, write_file, cls=NumpyArrayEncoder)
print("Done writing serialized NumPy array into file")
# Deserialization
print("Started Reading JSON file")
with open("numpyData.json", "r") as read_file:
print("Converting JSON encoded data into Numpy array")
decodedArray = json.load(read_file, cls=NumpyArrayDecoder)
print("Re-Imported Data: \n")
print(decodedArray)
