Minimal way to set PYTHONPATH for developing python in VS Code and Jupyter notebook server

Question

Related questions that do not apply:

Set PYTHONPATH for local Jupyter Notebook in VS Code - does not apply to running Jupyter server.
How to set pythonpath at startup when running jupyter notebook server - no answer
Python: Module Not Found in Jupyter Notebook - solution involves absolute path
Python module not found <project> only on VPS. works fine on local machine - solution involves absolute path and modifying every notebook

I'm helping to organize some python code with a team at work. For "reasons" the project has multiple subdirectories and so needs PYTHONPATH or equivalent to add those subdirectories to work. For example, with this project setup…

project/
   .venv/
   foo/
      bar.py
   jim/
      jam.py

…we want Jupyter notebooks to work for the code:

import jim
import jam

We want these notebooks to work both (a) within VS Code, and (b) when starting a Jupyter notebook server and connecting to it from Google Colab.

Desires:

Little-to-no setup work needed per user. (Ideally, they just clone the project and run a launch command or manual script from within the .venv to get Jupyter server running.)
No absolute paths hard-coded anywhere. (Multiple copies of the project sometimes exist on disk, are added and removed; a user needs to be able to launch Jupyter notebook servers for various locations.)
Specify the foo and bar paths in as few places as possible; ideally one. (There are many, many places where extra paths can be specified within the ecosystem of Python, Jupyter, VS Code, pytest, pylint, etc.)
No need to pollute every single notebook file by adding sys.path modifications at the top of each.
Live development is possible. The developers are not installing this project and then using the notebooks, they are iteratively using the notebooks and modifying core library code that affects the next notebook evaluation.

Within VS Code I've gotten the notebooks working by:

Adding .env file at the project root with the contents:
```
PYTHONPATH=foo:bar
```
- The Python extension has a default setting:
  "python.envFile" : "${workspaceFolder}/.env"
Ensuring all notebooks are run (in VS Code) from the root directory by changing the setting jupyter.notebookFileRoot from the default ${fileDirname} to ${workspaceFolder}.

I've also had to add the locations in the pyproject.toml file for testing:

[tool.pytest.ini_options]
pythonpath = [
    "foo",
    "bar",
]

Now…how can I make it so that all the developers can launch a Jupyter notebook server for the project, inside the project's venv, that gets PYTHONPATH set correctly? It can be a single launch_jupyter.sh script, but I'd prefer not to have to maintain the same list of foo:bar in that script file. (Because it's not DRY, and the actual directory names are longer and more than just a couple.)

Phrogz · Accepted Answer · 2025-03-05 14:50:28Z

I ended up writing a shell script that:

Activates (or creates) the virtual environment, using uv.
Loads environment variables from the same .env file that VSCode uses
Converts all the relative paths in PYTHONPATH to absolute
Starts the Jupyter server.

Now developers can just ./start-jupyter-server.sh and then connect to it from Google Colab.

#!/bin/bash

# Find the directory of this file
SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" &> /dev/null && pwd)

# Path to the desired virtual environment
DESIRED_VENV="$SCRIPT_DIR/.venv"

# Check if a virtual environment is active
if [ -n "$VIRTUAL_ENV" ]; then
  # If active, check if it's the desired one
  if [ "$VIRTUAL_ENV" != "$DESIRED_VENV" ]; then
    echo "Deactivating virtual environment $VIRTUAL_ENV because it is not $DESIRED_VENV..."
    deactivate
  fi
fi

# Activate the desired virtual environment
if [ ! -d "$DESIRED_VENV" ]; then
  echo "Virtual environment not found at $DESIRED_VENV; creating."
  uv venv
fi

echo "Activating virtual environment..."
source "$DESIRED_VENV/bin/activate"
uv sync

# Load environment variables from .env file
if [ -f "$SCRIPT_DIR/.env" ]; then
  # Enable exporting variables from .env file
  set -a
  # Source the .env file
  source "$SCRIPT_DIR/.env"
  # Disable exporting variables
  set +a
else
  echo "Error: .env file not found."
  exit 1
fi

# Convert relative paths to absolute paths
PYTHONPATH=$(echo "$PYTHONPATH" | tr ':' '\n' | while read -r path; do
  echo -n "$SCRIPT_DIR/$path:"
done | sed 's/:$//')

# Set default port if not provided
PORT=${1:-8888}

# Run Jupyter notebook
jupyter notebook \
  --NotebookApp.allow_origin='https://colab.research.google.com' \
  --Application.log_level=50 \
  --port=$PORT \
  --NotebookApp.port_retries=0 \
  --no-browser

Collectives™ on Stack Overflow

Minimal way to set PYTHONPATH for developing python in VS Code and Jupyter notebook server

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related