translating bash script to python

Question

I have some simple bash scripts that I would like to translate in python. I need to define some bash environment variables before calling the external program with subprocess. In particular I need to load a module environment. The original bash looks something like this

#!/bin/bash 
source /etc/profile.d/modules.sh
module purge
module load intel

./my_code

I want to call `my_code` with subprocess, but how do I pass the variables defined by `module` to the shell?

Edit:

my first attempt to convert the above script is

#!/usr/bin/python -tt
import subprocess

subprocess.call("source /etc/profile.d/modules.sh",shell=True)
subprocess.call("module purge",shell=True)
subprocess.call("module load intel",shell=True)
subprocess.call("./mycode",shell=True)

but this fails, because the environment variables (that I think are LIBRARY_PATH and PATH and C_INCLUDE) are modified in the shell launched by subprocess, but then this shell dies and these variables are not inherited by subsequent shells. So the question is: how can I launch a modules command and then save the relevant variables with os.environ ?

So what is your question?

slider
– slider

2014-03-19 14:21:57 +00:00
Commented Mar 19, 2014 at 14:21 — slider
– slider, Commented Mar 19, 2014 at 14:21
I think now the question is clear, isn't it?

simona
– simona

2014-03-19 14:32:08 +00:00
Commented Mar 19, 2014 at 14:32 — simona
– simona, Commented Mar 19, 2014 at 14:32

Community · Accepted Answer · 2017-05-23 12:28:48Z

1

You could use a multiline string so that the environment is modified in the same shell process:

from subprocess import check_call

check_call("""
source /etc/profile.d/modules.sh
module purge
module load intel

./my_code
""", shell=True, executable="/bin/bash")

See also Calling the “source” command from subprocess.Popen

edited May 23, 2017 at 12:28

CommunityBot

11 silver badge

answered Mar 20, 2014 at 15:31

jfs

417k210 gold badges1k silver badges1.7k bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

simona Over a year ago

yeah, this is certainly a solution. but you know the above code is a simplification, between module load and the call to mycode I do some string manipulation and I actually call several programs that use the same modules, so this would not be very elegant

jfs Over a year ago

@simona: what do you mean: "would not be very elegant"?

simona Over a year ago

it seemed cleaner to me to pass the environment variables from one shell to the other, like you would do in bash with export, there must be a way to do that.

jfs Over a year ago

@simona: Why do you need several shell processes? Also, check out the link that I've added above

scav · Accepted Answer · 2014-03-19 14:45:53Z

0

import os

os.environ is a dict of environment variables (exported shell variables)

answered Mar 19, 2014 at 14:45

scav

1,1551 gold badge9 silver badges17 bronze badges

1 Comment

simona Over a year ago

I modified my question

Collectives™ on Stack Overflow

translating bash script to python

I want to call `my_code` with subprocess, but how do I pass the variables defined by `module` to the shell?

2 Answers 2

4 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

I want to call my_code with subprocess, but how do I pass the variables defined by module to the shell?

2 Answers 2

4 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related

I want to call `my_code` with subprocess, but how do I pass the variables defined by `module` to the shell?