Using numpy.array with large number of dimensions

Question

If you try to make an array with large number of dimensions in numpy, it throws an exception:

In [1]: import numpy as np

In [2]: a = np.zeros((1,) * 33)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-32dc30f6e439> in <module>()
----> 1 a = np.zeros((1,) * 33)

ValueError: sequence too large; must be smaller than 32

Are there any simple workarounds to this?

What are the reasons why numpy doesn't allow creating such arrays?

If you have that many dimensions, either you're going to have a ton of length-1 dimensions you could probably remove, or you're going to have trouble fitting that much array into RAM. numpy.zeros([2]*33) wouldn't even fit in a 32-bit address space. — user2357112
– user2357112, Commented Feb 4, 2016 at 4:37
@user2357112 Yes, I have a lot of length-1 dimensions, and no, I can't remove them — DLunin
– DLunin, Commented Feb 4, 2016 at 4:42
@user2357112 Having these dimensions simplifies many operations, and (the main reason) a lot of code already relies on it — DLunin
– DLunin, Commented Feb 4, 2016 at 4:45

Jaime · Accepted Answer · 2016-02-04 04:50:53Z

12

From the NumPy source code:

/*
 * There are several places in the code where an array of dimensions
 * is allocated statically.  This is the size of that static
 * allocation.
 *
 * The array creation itself could have arbitrary dimensions but all
 * the places where static allocation is used would need to be changed
 * to dynamic (including inside of several structures)
 */

#define NPY_MAXDIMS 32
#define NPY_MAXARGS 32

You could change those defines and build from source a non-compatible version that fits your needs.

answered Feb 4, 2016 at 4:50

Jaime

67.7k19 gold badges128 silver badges164 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Using numpy.array with large number of dimensions

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related