How to create a simple numpy array using the C API, with proper memory management?

Question

I have two related questions regarding creating a numpy array using the C API. Given a newly created numpy array:

    std::vector<double> vec({0.1, 0.2});
    int length = vec.size();
    double* data = new double[length];
    std::copy(vec.begin(), vec.end(), data); 
    PyObject* obj = PyArray_SimpleNewFromData(1, &length, NPY_DOUBLE, (void*)data);

How do I ensure proper memory management?

I didn't want to give PyArray_SimpleNewFromData a pointer to vec.size() since that's owned by vec so I copied the data into newly allocated memory. However, will numpy/python "just know" it needs deleting at end of scope? The docs mention about setting the OWNDATA flag, is this appropriate for heap-allocated memory?
How do I get a PyArrayObject* from a PyObject* returned from PyArray_SimpleNewFromData?

All the new array creation mechanisms such as PyArray_SimpleNewFromData return a PyObject*, but if you want to do anything using the numpy c api, you need a PyArrayObject*.

Edit

I was able to do an reinterpret_cast for 2, for instance:

    int owned = PyArray_CHKFLAGS(reinterpret_cast<PyArrayObject *>($result), NPY_ARRAY_OWNDATA);
    PyArray_ENABLEFLAGS(reinterpret_cast< PyArrayObject*>($result), NPY_ARRAY_OWNDATA);
    owned = PyArray_CHKFLAGS(reinterpret_cast< PyArrayObject *>($result), NPY_ARRAY_OWNDATA);
    if (!owned){
        throw std::logic_error("PyArrayObject does not own its memory");
    }

Still not sure if this is the "right" way to do it.

voidpointer · Accepted Answer · 2021-07-18 06:49:25Z

1

Regarding 1:

If the data ownership should be passed to the PyObject returned by PyArray_SimpleNewFromData then OWNDATA must be set. In this case the memory must be heap allocated, because stack allocated variables are released as soon as the scope of the variable is left. PyObject contains a reference count to manage the underlying memory. When the reference count is decremented to 0 then the object is deleted (Py_INCREF,Py_DECREF).

Regarding 2:

PyArrayObject is a struct which contains the variable ob_base of type PyObject. This PyObject is returned from PyArray_SimpleNewFromData. Since ob_base is the first variable of PyArrayObject, reinterpret_cast can be used to obtain PyArrayObject from PyObject. Py_TYPE can be used to check the type of a PyObject.

edited Jul 18, 2021 at 6:49

answered Jul 16, 2021 at 14:00

voidpointer

364 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

CiaranWelsh Over a year ago

I tried PyArrayObject * arr = Py_TYPE(obj); which errors with error C2440: 'initializing': cannot convert from 'PyTypeObject *' to 'PyArrayObject *'. Any ideas?

CiaranWelsh Over a year ago

I wasn't able to obtain a pointer to PyArrayObject using Py_TYPE, I was however able to do PyArrayObject* arr = reinterpret_cast<PyArrayObject*>(obj);

Collectives™ on Stack Overflow

How to create a simple numpy array using the C API, with proper memory management?

Edit

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

Edit

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related