Bash scripting from python [duplicate]

Question

I am confused about invoking bash from python code.
I wrote a small script that shows strange behaviors when invoking bash from python :

import subprocess

def check_output ( command,**w ):
    print( '\n>>> %s'%command )
    try :
        return subprocess.check_output( command,**w ).decode()
    except Exception as e :
        return str( e )

print( check_output( ['ls','-l']))
print( check_output( ['ls','-l',';','ls','-a']))
print( check_output( 'ls -l' ))
print( check_output( 'ls -l ; ls -a' ))
print( check_output( ['echo','99'],shell=1))
print( check_output( ['echo','11',';','echo','22'],shell=1))
print( check_output( 'echo 33 ; echo 44',shell=1 ))

And the output:

>>> ['ls', '-l']
total 124
-rwxrwxrwx 1 root root    59 Mar 22 09:57 main.py
-rwxrwxrwx 1 root root  3885 Mar 17 17:30 README.md
drwxrwxrwx 0 root root  4096 Mar 22 10:33 src
-rwxrwxrwx 1 root root    36 Mar 22 09:57 sub.c
drwxrwxrwx 0 root root  4096 Mar 22 10:33 test


>>> ['ls', '-l', ';', 'ls', '-a']
ls: cannot access ';': No such file or directory
ls: cannot access 'ls': No such file or directory
Command '['ls', '-l', ';', 'ls', '-a']' returned non-zero exit status 2.

>>> ls -l
[Errno 2] No such file or directory: 'ls -l': 'ls -l'

>>> ls -l ; ls -a
[Errno 2] No such file or directory: 'ls -l ; ls -a': 'ls -l ; ls -a'

>>> ['echo', '99']

>>> ['echo', '11', ';', 'echo', '22']

>>> echo 33 ; echo 44
33
44

The first and last commands are as I expected, but the others seems strange to me.
The multiple commands separator ; seems to act differently depending on parameter shell. And also the concatenated commands...

By the way I get the same results with python 2 and 3 (under kali linux).

Note that using shell=True is generally a bad idea. First, but less importantly, it's inefficient: Why start a copy of sh and tell it to start gcc, when you could instead just start gcc directly? Second, and more importantly, it raises security risks: When a shell is involved you need to worry if some joker told your program to save the compiled file as $(rm -rf ~).o; but without a shell, shell syntax in your data is harmless. — Charles Duffy
– Charles Duffy, Commented Mar 18, 2021 at 18:30
As for your question, though -- the real issue is identical to that in can a shell script set environment variables of the calling shell?; the difference (that there the parent process is a shell and here it's a Python interpreter) is immaterial to the problem. — Charles Duffy
– Charles Duffy, Commented Mar 18, 2021 at 18:34

Tim Roberts · Accepted Answer · 2021-03-18 18:27:59Z

1

When you launch a new process, that process gets a copy of your current environment, but any changes that it makes are only to its copy. When the process ends, that environment is thrown away. What that means is you can say in Python:

os.environ{'LD_LIBRARY_PATH'] = "/home/user/lib"

and all of the processes you start from then on will inherit that variable. But EACH copy of execute starts over with a fresh copy of your environment. Whatever changes they make do not survive.

answered Mar 18, 2021 at 18:27

Tim Roberts

55.3k4 gold badges28 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Bash scripting from python [duplicate]

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related