Python C API, send a python function pointer to c and execute it

Question

I want to create a function in python, pass it's function pointer to c and execute it there.

So my python file:

import ctypes
import example

def tester_print():
   print("Hello")

my_function_ptr = ctypes.CFUNCTYPE(None)(tester_print)
example.pass_func(my_function_ptr)

And here is what my function in c looks like:

typedef void (*MyFunctionType)(void);

PyObject* pass_func(PyObject *self, PyObject* args)
{
    PyObject* callable_object;
    if (!PyArg_ParseTuple(args, "O", &callable_object))
        return NULL;

    if (!PyCallable_Check(callable_object)) 
    {
        PyErr_SetString(PyExc_TypeError, "The object is not a callable function.");
        return NULL;
    }

    PyObject* function_pointer = PyCapsule_New(callable_object, "my_function_capsule", NULL);

    if (function_pointer == NULL) return NULL;

    MyFunctionType my_function = (MyFunctionType) PyCapsule_GetPointer(function_pointer, "my_function_capsule");

    if (my_function == NULL) return NULL;

    my_function(); // Or (*my_function)() Both same result.

    // PyCapsule_Free(function_pointer);

    Py_RETURN_NONE;
}

Doing this causes a seg fault on my_function() call. How can I do this?

Are you just trying to send a Python function? In that case, there's no need for ctypes function pointer shenanigans; just call the object via e.g. PyObject_Call. — nneonneo
– nneonneo, Commented Dec 23, 2022 at 12:50
@nneonneo I'm going to do some operations with, but not limited to, calling the function and using it's pointer to set threads and such. — Turgut
– Turgut, Commented Dec 23, 2022 at 12:54
If you're attempting to start a thread using a Python function, I'd recommend creating a C function to start the thread and pass it the Python callable (most thread libraries let you specify an argument for the new thread). — nneonneo
– nneonneo, Commented Dec 23, 2022 at 12:57
@nneonneo Yes that's exactly what I'm trying to do! Can you please embed that into my answer so I don't do something wrong again? — Turgut
– Turgut, Commented Dec 23, 2022 at 13:11

nneonneo · Accepted Answer · 2022-12-23 13:59:14Z

3

If you're just trying to pass a Python function to a C extension, pass it directly (don't use ctypes) and use PyObject_Call to call it:

example.pass_func(tester_print)

and

PyObject_CallNoArgs(callable_object);

If you need a real C function pointer for whatever reason, the usual approach is to write a C wrapper that takes the callable as an argument:

void callable_wrapper(PyObject *func) {
    PyObject_CallNoArgs(func);
    // plus whatever other code you need (e.g. reference counting, return value handling)
}

Most reasonable C APIs that take a callback function also provide a way to add an arbitrary argument to the callable ("user data"); for example, with pthreads:

result = pthread_create(&tid, &attr, callable_wrapper, callable_object);

Make sure to handle reference counting correctly: increment the reference on your callable object before passing it to the C API, and decrement the reference when it is no longer needed (e.g. if the callback is only called once, the callable_wrapper could DECREF before returning).

When using threads, you additionally need to ensure that you hold the GIL when calling any Python code; see https://docs.python.org/3/c-api/init.html#non-python-created-threads for more details and a code sample.

What your current code is doing is receiving a pointer to a ctypes CFUNCTYPE object as callable_object, placing that pointer in a capsule, taking it back out again, and calling it as if it was a C function pointer. This doesn't work, since it effectively attempts to call the CFUNCTYPE object as if it were a C function (the capsule stuff winds up being useless). When you're using the Python C API, there's almost never any need for ctypes in Python, because the C API can directly interact with Python objects.

edited Dec 23, 2022 at 13:59

answered Dec 23, 2022 at 12:55

nneonneo

181k37 gold badges331 silver badges412 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Turgut Over a year ago

Will this work on std::thread aswell? I've put the PyObject_CallNoArgs call inside a lambda and will attempt to do it that way.

nneonneo Over a year ago

Yes, you can pass the argument separately (e.g. std::thread(callable_wrapper, callable_object)), but of course there are many other ways to construct a suitable callable for std::thread in C++.

DavidW Over a year ago

You need to make sure that you hold the GIL inside your thread when you call your Python function.

nneonneo Over a year ago

@DavidW thanks for the reminder; I added it to my answer.

Collectives™ on Stack Overflow

Python C API, send a python function pointer to c and execute it

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related