16

Python's access to environment variables does not accurately reflect the operating system's view of the processes environment.

os.getenv and os.environ do not function as expected in particular cases.

Is there a way to properly get the running process' environment?


To demonstrate what I mean, take the two roughly equivalent programs (the first in C, the other in python):

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main(int argc, char *argv[]){
    char *env;
    for(;;){
        env = getenv("SOME_VARIABLE");
        if(env)
            puts(env);
        sleep(5);
    }
}

import os
import time
while True:
    env = os.getenv("SOME_VARIABLE")
    if env is not None:
        print env
    time.sleep(5)

Now, if we run the C program and attach to the running process with gdb and forcibly change the environment under the hood by doing something like this:

(gdb) print setenv("SOME_VARIABLE", "my value", 1)
[Switching to Thread -1208600896 (LWP 16163)]
$1 = 0
(gdb) print (char *)getenv("SOME_VARIABLE")
$2 = 0x8293126 "my value"

then the aforementioned C program will start spewing out "my value" once every 5 seconds. The aforementioned python program, however, will not.

Is there a way to get the python program to function like the C program in this case?

(Yes, I realize this is a very obscure and potentially damaging action to perform on a running process)

Also, I'm currently using python 2.4, this may have been fixed in a later version of python.

1
  • For what it's worth, this isn't unexpected: the library reference for the os module highlights the issue. Commented Oct 24, 2008 at 22:35

5 Answers 5

17

That's a very good question.

It turns out that the os module initializes os.environ to the value of posix.environ, which is set on interpreter start up. In other words, the standard library does not appear to provide access to the getenv function.

That is a case where it would probably be safe to use ctypes on unix. Since you would be calling an ultra-standard libc function.

Sign up to request clarification or add additional context in comments.

Comments

13

You can use ctypes to do this pretty simply:

>>> from ctypes import CDLL, c_char_p
>>> getenv = CDLL("libc.so.6").getenv
>>> getenv.restype = c_char_p
>>> getenv("HOME")
'/home/glyph'

Comments

4

Another possibility is to use pdb, or some other python debugger instead, and change os.environ at the python level, rather than the C level. Here's a small recipe I posted to interrupt a running python process and provide access to a python console on receiving a signal. Alternatively, just stick a pdb.set_trace() at some point in your code you want to interrupt. In either case, just run the statement "import os; os.environ['SOME_VARIABLE']='my_value'" and you should be updated as far as python is concerned.

I'm not sure if this will also update the C environment with setenv, so if you have C modules using getenv directly you may have to do some more work to keep this in sync.

Comments

3

I don't believe many programs EVER expect to have their environment externally modified, so loading a copy of the passed environment at startup is equivalent. You have simply stumbled on an implementation choice.

If you are seeing all the set-at-startup values and putenv/setenv from within your program works, I don't think there's anything to be concerned about. There are far cleaner ways to pass updated information to running executables.

Comments

1

Looking at the Python source code (2.4.5):

  • Modules/posixmodule.c gets the environ in convertenviron() which gets run at startup (see INITFUNC) and stores the environment in a platform-specific module (nt, os2, or posix)

  • Lib/os.py looks at sys.builtin_module_names, and imports all symbols from either posix, nt, or os2

So yes, it gets decided at startup. os.environ is not going to be helpful here.

If you really want to do this, then the most obvious approach that comes to mind is to create your own custom C-based python module, with a getenv that always invokes the system call.

1 Comment

Or I could use the ctypes module, but that just ruins the fun of it now, doesn't it?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.