Is there any easy way to debug cgi python programs apart from looking at the log file each time the browser generates an error?
4 Answers
You can use the cgitb module. It's as simple as
import cgitb
cgitb.enable()
It doesn't always work (e.g. it won't help you for permission errors, or certain other kinds of errors), but when it does it's quite helpful!
1 Comment
Martlark
This module only seems to offer enhanced trace on unhandled exceptions. It is due to be removed from the standard Python library with version 3.13
If you want to have an interactive debugging session where you can set breakpoints, examine variables, etc. set the environment variable QUERY_STRING to the desired value before calling cgi.FieldStorage(), and then execute the file as an ordinary script using your favorite debugger.
This shows the general idea:
>>> import cgi
>>> import os
>>> os.environ['QUERY_STRING'] = 'name=Bob&city=Toronto&favorite=chocolate'
>>> args = cgi.FieldStorage()
>>> args.keys()
['name', 'city', 'favorite']
Comments
You could capture (or form by hand) the data and env variables which the CGI script receives, then plainly run the script under your favorite debugger and feed the data to it.
In order to capture the incoming data you can just dump it from the script in CGI mode to some log file, then re-use under debugger in standalone mode.
Comments
Emulating CGI execution
You can emulate a CGI session by running the file as usual in your IDE and passing the environment variables that CGI expects. In your IDE find the launcher option to set custom environment variables, e.g. in pycharm under Environment variables set something like:
REQUEST_METHOD=GET;SERVER_PROTOCOL=HTTP/1.1;HTTP_HOST=localhost;REQUEST_URI="/cgi-bin/myscrpt.py?query=dinsdale&id=12345#"
Following the same approach you can also run a CGI script directly from the command line with:
REQUEST_METHOD=GET SERVER_PROTOCOL=HTTP/1.1 HTTP_HOST=localhost REQUEST_URI="/cgi-bin/myscrpt.py?query=dinsdale&id=12345#" python myscript.py
Debugging under a real server
To debug under a real server with an IDE such as pycharm or vscode you can use Run → Attach to process where you will have to select your CGI script.
There is a slight problem — CGI processes are single shot therefore you will need to manually suspend your script before it starts running.
One solution is to suspend your script before execution, based on some condition that you control, for example, the existence of a 'sentinel' file.
Example Sequence:
- At the top of your CGI script (assuming python3) add the following:
from pathlib import Path
while Path('/tmp/1').exists():
time.sleep(1)
- Run in terminal:
touch /tmp/1 - Call the CGI script in your browser — the python interpreter will fire up, the script will go into sleep, and the page will hang in loading state.
- In PyCharm set a breakpoint in your CGI script, and then pick on the menu Run → Attach to process and select the name of your cgi script (in VsCode create a debug configuration of type Attach Using Process Id and pick your process).
- Run in terminal:
rm /tmp/1. - The
sleeploop will exit, and PyCharm/VsCode will now stop at the breakpoint.
NOTE: In order to debug, your script must be part of the current python project.
In PyCharm (CE or Pro) the two commands to create and remove the temporary file can be added as external tools and then they can be added as toolbar buttons which will make the process easier (first, add the commands under Preferences > Tools > External Tools, and then right-click the toolbar and pick Customize menus and toolbars and add the actions). See also https://www.jetbrains.com/help/pycharm/customize-actions-menus-and-toolbars.html. In VsCode the same can be achieved via plugins such as Commands.
tailon the log, so I keep an eye on that console.