How to run a shell script in python from memory?

Question

The application I'm writing retrieves a shell script through HTTP from Network, I want to run this script in python however I don't want to physically save it to the hard drive because I have its content already in memory, and I would like to just execute it. I have tried something like this:

import subprocess

script = retrieve_script()
popen = subprocess.Popen(scrpit, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
stdOut, stdErr = popen.communicate()

def retrieve_script_content():
    # in reality I retrieve a shell script content from network,
    # but for testing purposes I will just hardcode some test content here
    return "echo command1" + "\n" + "echo command2" + " \n" + "echo command3"

This snippet will not work because subprocess.Popen expects you to provide only one command at a time.

Are there any alternatives to run a shell script from memory?

Is this maybe a duplicate of How to run multiple commands synchronously from one subprocess.Popen command?? — YellowShark
– YellowShark, Commented Oct 18, 2018 at 14:18

that other guy · Accepted Answer · 2018-10-18 18:26:51Z

3

This snippet will not work because subprocess.Popen expects you to provide only one command at a time.

That is not the case. Instead, the reason why it doesn't work is:

The declaration of retrieve_script has to come before the call
You call it retrieve_script_content instead of retrieve_script
You misspelled script as scrpit

Just fix those and it's fine:

import subprocess

def retrieve_script():
    return "echo command1" + "\n" + "echo command2" + " \n" + "echo command3"

script = retrieve_script()
popen = subprocess.Popen(script, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
stdOut, stdErr = popen.communicate()
print(stdOut);

Result:

$ python foo.py
command1
command2
command3

However, note that this will ignore the shebang (if any) and run the script with the system's sh every time.

edited Oct 18, 2018 at 18:26

answered Oct 18, 2018 at 18:14

that other guy

125k12 gold badges187 silver badges214 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Mykhailo Seniutovych Over a year ago

It was a typo, but I posted a test code sample here. I can confirm that running bash scripts like that indeed works on Linux. However when I ran similar code on Windows it did not work, so you will not be able to run batch files like that on Windows.

unutbu · Accepted Answer · 2018-10-19 11:45:48Z

2

Are you using a Unix-like OS? If so, you should be able to use a virtual filesystem to make an in-memory file-like object at which you could point subprocess.Popen:

import subprocess
import tempfile
import os
import stat

def retrieve_script_content():
    # in reality I retrieve a shell script content from network,
    # but for testing purposes I will just hardcode some test content here
    return "echo command1" + "\n" + "echo command2" + " \n" + "echo command3"

content = retrieve_script_content()
with tempfile.NamedTemporaryFile(mode='w', delete=False, dir='/dev/shm') as f:
    f.write(content)
    os.chmod(f.name, stat.S_IRUSR | stat.S_IXUSR)
    # print(f.name)
popen = subprocess.Popen(f.name, stdout=subprocess.PIPE, stderr=subprocess.PIPE, 
                         shell=True)
stdOut, stdErr = popen.communicate()

print(stdOut.decode('ascii'))
# os.unlink(f.name)

prints

command1
command2
command3

Above I used /dev/shm as the virtual filesystem since Linux systems based on Glibc always have a tmpfs mounted on /dev/shm. If security is a concern you may wish to setup a ramfs.

One reason why you might want to use a virtual file instead of passing the script contents directly to subprocess.Popen is that the maximum size for a single string argument is limited to 131071 bytes.

edited Oct 19, 2018 at 11:45

answered Oct 18, 2018 at 14:47

unutbu

886k197 gold badges1.9k silver badges1.7k bronze badges

1 Comment

Mykhailo Seniutovych Over a year ago

Thanks, the code will be running on Raspberry Pi Rasbian Jessie OS.

Serge · Accepted Answer · 2018-10-24 01:44:50Z

0

You can execute multi command script with Popen. Popen only restricts you to one-command string when shell flag is False, yet it is possible to pass a list of commands. Popen's flag shell=True allows for multi-command scripts (it is considered unsecure, though what you are doing - executing scripts from the web - is already very risky).

edited Oct 24, 2018 at 1:44

answered Oct 18, 2018 at 14:16

Serge

3,8453 gold badges20 silver badges39 bronze badges

2 Comments

Mykhailo Seniutovych Over a year ago

We have a web service that provides our application with shell scripts so I know what I'm doing no worries :)

Serge Over a year ago

The answer address general question of how one can execute a downloaded multiline script, though indeed does not goes to reviewing the sample code.

Collectives™ on Stack Overflow

How to run a shell script in python from memory?

3 Answers 3

1 Comment

1 Comment

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related