call an executable string from python

Question

I'm trying to find a way to run an executable script that can be downloaded from the web from Python, without saving it as a file. The script can be python code or bash or whatever - it should execute appropriately based on the shebang. I.e. if the following were saved in a file called script, then I want something that will run ./script without needing to save the file:

#!/usr/bin/env python3
import sys
from my_module import *

scoped_hash = sys.argv[1]
print(scoped_hash)

I have a function that reads such a file from the web and attempts to execute it:

def execute_artifact(command_string):
    os.system('sh | ' + command_string)

Here's what happens when I call it:

>>> print(string)
'#!/usr/bin/env python3\nimport sys\nfrom my_module import *\n\nscoped_hash = sys.argv[1]\n\nobject_string = read_artifact(scoped_hash)\nparsed_object = parse_object(object_string)\nprint(parsed_object)\n'
>>> execute_artifact(string) 
sh-3.2$ Version: ImageMagick 7.0.10-57 Q16 x86_64 2021-01-10 https://imagemagick.org
Copyright: © 1999-2021 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI Modules OpenMP(4.5) 
Delegates (built-in): bzlib freetype gslib heic jng jp2 jpeg lcms lqr ltdl lzma openexr png ps tiff webp xml zlib
Usage: import [options ...] [ file ]

Bizarrely, ImageMagick is called. I'm not sure what's going on, but I'm sure there's a better way to do this. Can anyone help me?

@juanpa.arrivillaga I guess because of the shebang telling it to interpret the code as python. — David J.
– David J., Commented May 3, 2021 at 5:46
OK, I wrote an answer assuming it was Python always. You want to dynamically execute some text based on the shebang line. Looking around, it seems like the kernel handles the shebang on linux. Note sure if there is any great ways to do this without using a file other than parsing the shebang yourself and manually invoking the appropriate interpreter. Related: askubuntu.com/questions/850384/… — juanpa.arrivillaga
– juanpa.arrivillaga, Commented May 3, 2021 at 6:03
@juanpa.arrivillaga. I'd be interested in seeing your answer assuming it's python, if you wouldn't mind posting that as an answer — David J.
– David J., Commented May 3, 2021 at 6:11

juanpa.arrivillaga · Accepted Answer · 2021-05-03 07:34:21Z

2

EDIT: This answer was added before OP updated requirements to include:

The script can be python code or bash or whatever - it should execute appropriately based on the shebang.

Some may still find the below helpful if they decided to try to parse the shebang themselves:

Probably, the sanest way to do this is to pass the string to the python interpreter as standard input:

import subprocess

p = subprocess.Popen(["python"], stdin=subprocess.PIPE)
p.communicate(command_string.encode())

My instinct tells me this entire thing is fraught with pitfalls. Perhaps, at least, you want to launch it using the same executable that launched your current process, so:

import subprocess
import sys

p = subprocess.Popen([sys.executable], stdin=subprocess.PIPE)
p.communicate(command_string.encode())

If you want to use arguments, I think using the -c option to pass in code as a string as an argument works, then you have access to the rest, so:

import subprocess
import sys

command_string = """
import sys
print(f"{sys.argv=}")
"""

completed_process = subprocess.run([sys.executable, "-c", command_string, "foo", "bar", "baz"])

The above prints:

sys.argv=['-c', 'foo', 'bar', 'baz']

edited May 3, 2021 at 7:34

answered May 3, 2021 at 5:54

juanpa.arrivillaga

97.6k14 gold badges141 silver badges190 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

David J. Over a year ago

How would I pass the input? I tried this ` p.communicate(command_string.encode(), 'alkwjefwfj')`. and it returned an error origininating in subprocess.py.

juanpa.arrivillaga Over a year ago

@DavidJ. why are you passing 'alkwjefwfj'??? That's being passed as the timeout parameter... You just need p.communicate(command_string.encode()) where command_string is your string... (it needs to be encoded to bytes but that's it)

David J. Over a year ago

command_string.encode() is a string of python code that expects an argument (it includes sys.argv. I'm wondering how this argument can be passed.

juanpa.arrivillaga Over a year ago

@DavidJ. hmm, found a way using the -c option, where you pass a string of python statements to be executed. Note sure if it's the best approach but it seems to work

dumbass · Accepted Answer · 2021-05-05 18:07:58Z

This cannot be done in full generality.

If you want the shebang line to be interpreted as usual, you must write the script to a file. This is a hard requirement of the protocol that makes shebangs work. When a script with a shebang line is executed by the operating system, the kernel (and yes, it’s not the shell which does it, unlike what the question implies) reads the script and invokes the interpreter specified in the shebang, passing the pathname of the script as a command line argument. For that mechanism to work, the script must exist in the file system where the interpreter can find it. (It’s a rather fragile design, leading to some security issues, but it is what it is.)

Many interpreters will allow you to specify the program text on standard input or on the command line, but it is nowhere guaranteed that it will work for any interpreter. If you know you are working with an interpreter which can do it, you can simply try to parse the shebang line yourself and invoke the interpreter manually:

import io
import subprocess
import re

_RE_SHBANG = re.compile(br'^#!\s*(\S+)(?:\s+(.*))?\s*\n$')

def execute(script_body):
    stream = io.BytesIO(script_body)
    shebang = stream.readline()
    m = _RE_SHBANG.match(shebang)
    if not m:
        # not a shebang
        raise ValueError(shebang)
    interp, arg = m.groups()
    arg = (arg,) if arg is not None else ()
    return subprocess.call([interp, *arg, '-c', script_body])

The above will work for POSIX shell and Python scripts, but not e.g. for Perl, node.js or standalone Lua scripts, as the respective interpreters take the -e option instead of -c (and the latter doesn’t even ignore shebangs in code given on the command line, so that needs to be separately stripped too). Feeding the script to the interpreter through standard input is also possible, but considerably more involved, and will prevent the script itself from using the standard input stream. That is also possible to overcome, but it doesn’t change the fact that it’s just a makeshift workaround that isn’t anywhere guaranteed to work in the first place. Better to simply write the script to a file anyway.

Collectives™ on Stack Overflow

call an executable string from python

2 Answers 2

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related