Tracing code execution in embedded Python interpreter

Question

I'd like to create an application with embedded python interpreter and basic debugging capabilities. Now I'm searching the API for functions which I could use to run code step-by-step and get the number of the current line of code which is being (or is about to be) executed.

Official Python docs seem a little underdone for me when comes it to tracing and profiling. There is, for example, no information about the meaning of the return value of Py_tracefunc.

So far I've assembled the following:

#include <Python.h>

static int lineCounter = 0;

int trace(PyObject *obj, PyFrameObject *frame, int what, PyObject *arg)
{
    if(what == PyTrace_LINE)
    {
        lineCounter += 1;
        printf("line %d\n", lineCounter);
    }
    return 0;
}

int main(int argc, char *argv[])
{
    wchar_t *program = Py_DecodeLocale(argv[0], NULL);
    if (program == NULL) {
        fprintf(stderr, "Fatal error: cannot decode argv[0]\n");
        exit(1);
    }
    Py_SetProgramName(program);  /* optional but recommended */
    Py_Initialize();
    PyEval_SetTrace(trace, NULL);
    char *code = "def adder(a, b):\n"
                 " return a + b\n"
                 "x = 3\n"
                 "y = 4\n"
                 "print(adder(x, y))\n";
    PyRun_SimpleString(code);
    Py_Finalize();
    PyMem_RawFree(program);
    return 0;
}

However, the compiler outputs the following error:

hello.c:5:26: error: unknown type name ‘PyFrameObject’
 int trace(PyObject *obj, PyFrameObject *frame, int what, PyObject *arg)
                          ^

I'm operating on ManjaroLinux and using the following to compile the above:

gcc -o hello hello.c -I/usr/include/python3.5m  -Wno-unused-result -Wsign-compare -Wunreachable-code -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong --param=ssp-buffer-size=4 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -L/usr/lib -lpython3.5m -lpthread -ldl  -lutil -lm  -Xlinker -export-dynamic

I've found that I can replace PyFrameObject with struct _frame and then program compiles but everyone knows it's a dirty hack, not a solution.

The executable outputs the following:

line 1
line 2
line 3
line 4
line 5
7

But I'd like the traces to follow the execution flow of the script (that is: start from line 3, then 4, 5 and then, due to the function call, 2).

I could not find anything about step-by-step execution.

Could you recommend some other sources about Python C API with more information and some introduction to the topic?

I awarded the answer with bounty since it would expire anyway. However, I'm still looking and would be grateful for answers for other questions from above.

This might help regarding the trace function: docs.python.org/3/library/sys.html#sys.settrace — Sven Marnach
– Sven Marnach, Commented Dec 3, 2015 at 20:56
All of the CPython source code, actually, not only the header files. You are working with implementation details of the CPython interpreter here, and in these cases, it's often necessary (o at least helpful) to look at the source code in addition to the documentation. E.g. see the definition of Py_tracefunc for the meaning of the return value. — Sven Marnach
– Sven Marnach, Commented Dec 4, 2015 at 11:26
Well, read the source code. Here's where the trace function is called: github.com/python/cpython/blob/…. You can look for the callers of this function (in the same file), and see how it behaves if you return an error (it won't enter the frame). If you don't want this to happen, you probably don't want to throw an exception. — Sven Marnach
– Sven Marnach, Commented Dec 4, 2015 at 12:08

Iharob Al Asimi · Accepted Answer · 2015-12-09 22:12:05Z

9

+50

hello.c:5:26: error: unknown type name ‘PyFrameObject’

This error means that PyFrameObject has not been declared. I did a Google search which showed me frameobject.h in the Python source tree is where that structure is declared.

I expect that you can add the line

#include <frameobject.h>

to resolve this.

edited Dec 9, 2015 at 22:12

Iharob Al Asimi

53.1k6 gold badges66 silver badges100 bronze badges

answered Dec 3, 2015 at 21:12

dsh

12.3k3 gold badges37 silver badges53 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Luke Over a year ago

I read somewhere on the Internet that Python.h alone should be sufficient. The example from the docs also includes only Python.h.

Luke Over a year ago

Could you provide any source claiming that frameobject.h should be included separately?

dsh Over a year ago

I inferred it from the error message you posted. You can determine for yourself by reading the source. Read Python.h and follow the various includes to see where/if it includes frameobject.h. It might even be conditional based on defined preprocessor tokens.

Luke Over a year ago

For that much I could just trust you. Too bad the docs are pretty selective. But that simple compilation error isn't really my biggest worry, as you can probably guess.

Xavier Combelle Over a year ago

@Luke Python.h should be sufficient. However, it could be a bug in python source code

|

infinite loop · Accepted Answer · 2017-08-10 20:05:11Z

0

The pyFrameObject has a

int f_lineno;

field. You can use it. But apparently, it is not always storing the correct value. So, you should probably use the function:

/* Return the line of code the frame is currently executing. */
int PyFrame_GetLineNumber(PyFrameObject *);

then, you can use

frame->f_code->co_filename

to get the current file name

frame->f_code->co_name

to get the current function name and

frame->f_back

to get one level down in the call stack. .

edited Aug 10, 2017 at 20:05

infinite loop

1,3299 silver badges20 bronze badges

answered Aug 10, 2017 at 15:03

Pierre Saunier

1

Comments

Community · Accepted Answer · 2022-11-02 04:31:18Z

0

According to docstring in Python v3.10.8 in Include/cpython/pystate.h:

/* Py_tracefunc return -1 when raising an exception, or 0 for success. */
typedef int (*Py_tracefunc)(PyObject *, PyFrameObject *, int, PyObject *);

edited Nov 2, 2022 at 4:31

CommunityBot

11 silver badge

answered Oct 27, 2022 at 15:43

Tanner Firl

1006 bronze badges

1 Comment

Community Over a year ago

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.

mnille · Accepted Answer · 2016-10-25 19:16:45Z

-1

PyFrameObject is just a _frame struct. Just replace PyFrameObject by _frame in your function signature and you won't have to include any additional python headers.

edited Oct 25, 2016 at 19:16

mnille

1,3584 gold badges16 silver badges22 bronze badges

answered Oct 25, 2016 at 17:01

John McKowen

1

Collectives™ on Stack Overflow

Tracing code execution in embedded Python interpreter

4 Answers 4

7 Comments

Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

7 Comments

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related