Undertanding python import process (importing custom os module)

Question

I was reading through the python docs for how import is resolved and found this

... the interpreter first searches for a built-in module with that name. These module names are listed in sys.builtin_module_names. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:

The directory containing the input script (or the current directory when no file is specified).

PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH). ...

So python first looks into sys.builtin_module_names and then into sys.path. So I checked sys.builtin_module_names on my OS (Mac).

>>> sys.builtin_module_names
('_abc', '_ast', '_codecs', '_collections', '_functools', '_imp', '_io', '_locale', '_operator', '_signal', '_sre', '_stat', '_string', '_suggestions', '_symtable', '_sysconfig', '_thread', '_tokenize', '_tracemalloc', '_typing', '_warnings', '_weakref', 'atexit', 'builtins', 'errno', 'faulthandler', 'gc', 'itertools', 'marshal', 'posix', 'pwd', 'sys', 'time')
>>> 'os' in sys.builtin_module_names
False

Since, os is not in sys.builtin_module_names, a file named os.py in the same directory as my python file should take precedence over the os python module.

I created a file named os.py in a test directory with the following simple code:

#os.py
def fun():
    print("Custom os called!")

And created another file named test.py which imports os

#test.py
import sys

print("os in sys.builtin_module_names -", 'os' in sys.builtin_module_names)
print("First directory on sys -", sys.path[0])

import os

print("source file of the imported os -", os.__file__)
print(os.fun())

This is the output of test.py

> python3 test.py 
os in sys.builtin_module_names - False
First directory on sys - /Users/dhruv/Documents/test
source file of the imported os - /opt/homebrew/Cellar/[email protected]/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/os.py
Traceback (most recent call last):
  File "/Users/dhruv/Documents/test/test.py", line 9, in <module>
    print(os.fun())
          ^^^^^^
AttributeError: module 'os' has no attribute 'fun'

Why is the python os module called?

user2357112 · Accepted Answer · 2025-02-07 16:30:44Z

1

sys.path is initialized the way the tutorial says, but Python needs the os module before that initialization can fully complete.

os is needed during Python setup. It's imported by the site module, which is imported during Python setup unless you manually say not to do that (with the -S flag).

These imports happen so early, the Python interpreter hasn't actually decided what Python program it's going to run yet. It figures that out later. And it can't add the script directory to sys.path until it's decided what the script is.

So at this early point in setup, the script directory isn't on sys.path, and imports don't search the script directory. Python can't find your os.py this early.

Now, on more recent Python versions, there's another thing to consider. On Python 3.11 and up, the devs decided to freeze a bunch of modules written in Python that are needed at Python startup. This embeds their bytecode directly into the Python executable, which helps make startup faster.

Frozen modules aren't mentioned in the tutorial - tutorials usually gloss over obscure details like this. Frozen modules are searched for after built-in modules and before the sys.path search. os is one of the modules that got frozen, so on Python 3.11 and up, the stdlib version of os would take priority over an os.py in the script directory even once sys.path is fully initialized.

edited Feb 7 at 16:30

answered Feb 7 at 11:11

user2357112

286k32 gold badges490 silver badges569 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Dhruv Feb 7 at 11:41

If it is imported already then why do we have to import it to use it?

Dhruv Feb 7 at 11:42

Also, if I run sys.modules.keys() at the start of the program. Would all the values in that behave the same as in the question? Or can we say because os was in sys.modules it is not imported from the directory?

Dhruv Feb 7 at 13:28

Hey, it would be great if you could help with the comments as well. Thanks for the answer anyway

user2357112 Feb 7 at 13:39

@Dhruv: That's not what Stack Overflow comments are supposed to be for - if you have more questions, you should treat them as new questions, researching them first and then using the "Ask Question" button if you can't find the answer on your own.

Dhruv Feb 7 at 13:50

I just thought that they were not big enough to be posted as separate questions. Will post them. Appreciate your help in telling me the "Ask Question" button.

|

Collectives™ on Stack Overflow

Undertanding python import process (importing custom os module)

1 Answer 1

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related