How to load multiple .npy files into a single numpy array

Question

I want to load multiple numpy file and putting them in array like this ["file1.npy","file2.npy","file3.npy",......] to apply pca dimensionality reduction on this array.

Any help would be appreciated

code

k=1
for indexPatient in range(0, len(patients)): 
    interictalData_withoutpca=np.concatenate((interictalData, tmpData[0:22,start*256:end]), axis=1)
    x=np.array(interictalData_withoutpca)
    y=np.save('interictalData_matrix'+str(k)+'_'+patients[indexPatient]+'_'+str(l),x)
    k+=1

Han-Kwang Nienhuys · Accepted Answer · 2020-06-21 18:45:50Z

9

The easiest way is:

filenames = ["file1.npy", "file2.npy", "file3.npy"]
combined_data = np.array([np.load(fname) for fname in filenames])

This requires that the arrays stored in each of the files have the same shape; otherwise you get an object array rather than a multidimensional array.

If it's a large amount of data and you know the shape of the data, say (n1, n2), it's more efficient (in speed and memory) to preallocate the data:

combined_data = np.zeros((len(filenames), n1, n2))
for i, fn in enumerate(filenames):
   combined_data[i, :, :] = np.load(fn)

Also, note that your code to generate the data can be made more Pythonic:

for k, patient in enumerate(patients, start=1):
    idata = np.concatenate((interictalData, tmpData[0:22, start*256:end]), axis=1)
    np.save(f'interictalData_matrix{k}_{patient}_{l}.npy', idata)

answered Jun 21, 2020 at 18:45

Han-Kwang Nienhuys

3,3242 gold badges15 silver badges33 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to load multiple .npy files into a single numpy array

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related