How to convert shell script in one line and execute it from Python subprocess?

Question

I have my below simple shell script which I am trying to convert in one single line so that I can execute it from my Python program using subprocess -

#!/bin/bash
set -e

COUNT=60   #number of 10 second timeouts in 10 minutes
SUM_SYNCS=0
SUM_SYNCS_BEHIND=0
HOSTNAME=$hostname

echo $HOSTNAME

while [[ $COUNT -ge "0" ]]; do

#send the request, put response in variable
DATA=$(wget -O - -q -t 1 http://$HOSTNAME:8080/beat)

#grep $DATA for syncs and syncs_behind
SYNCS=$(echo $DATA | grep -oE 'num_syncs: [0-9]+' | awk '{print $2}')
SYNCS_BEHIND=$(echo $DATA | grep -oE 'num_syncs_behind: [0-9]+' | awk '{print $2}')

echo $SYNCS
echo $SYNCS_BEHIND

#verify conditionals
if [[ $SYNCS -gt "8" && $SYNCS_BEHIND -eq "0" ]]; then exit 0; fi

#decrement the counter
let COUNT-=1

#wait another 10 seconds
sleep 10

done

I converted the above script in one line like this -

jsonStr = {"script": "#!/bin/bash\nset -e\n\nCOUNT=60   #number of 10 second timeouts in 10 minutes\nSUM_SYNCS=0\nSUM_SYNCS_BEHIND=0\nHOSTNAME=$hostname\t\n\necho $HOSTNAME\n\nwhile [[ $COUNT -ge \"0\" ]]; do\n\n#send the request, put response in variable\nDATA=$(wget -O - -q -t 1 http://$HOSTNAME:8080/beat)\n\n#grep $DATA for syncs and syncs_behind\nSYNCS=$(echo $DATA | grep -oE 'num_syncs: [0-9]+' | awk '{print $2}')\nSYNCS_BEHIND=$(echo $DATA | grep -oE 'num_syncs_behind: [0-9]+' | awk '{print $2}')\n\necho $SYNCS\necho $SYNCS_BEHIND\n\n#verify conditionals\nif [[ $SYNCS -gt \"8\" && $SYNCS_BEHIND -eq \"0\" ]]; then exit 0; fi\n\n#decrement the counter\nlet COUNT-=1\n\n#wait another 10 seconds\nsleep 10\n\ndone\n"}

And I am trying to execute this using Python subprocess but everytime I am getting an error -

  File "C:\Python27\lib\json\__init__.py", line 326, in loads
    return _default_decoder.decode(s)
  File "C:\Python27\lib\json\decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer

Below is my full Python code which I am trying out -

jsonStr = {"script": "#!/bin/bash\nset -e\n\nCOUNT=60   #number of 10 second timeouts in 10 minutes\nSUM_SYNCS=0\nSUM_SYNCS_BEHIND=0\nHOSTNAME=$hostname\t#\n\necho $HOSTNAME\n\nwhile [[ $COUNT -ge \"0\" ]]; do\n\n#send the request, put response in variable\nDATA=$(wget -O - -q -t 1 http://$HOSTNAME:8080/beat)\n\n#grep $DATA for syncs and syncs_behind\nSYNCS=$(echo $DATA | grep -oE 'num_syncs: [0-9]+' | awk '{print $2}')\nSYNCS_BEHIND=$(echo $DATA | grep -oE 'num_syncs_behind: [0-9]+' | awk '{print $2}')\n\necho $SYNCS\necho $SYNCS_BEHIND\n\n#verify conditionals\nif [[ $SYNCS -gt \"8\" && $SYNCS_BEHIND -eq \"0\" ]]; then exit 0; fi\n\n#decrement the counter\nlet COUNT-=1\n\n#wait another 10 seconds\nsleep 10\n\ndone\n"}

j = json.loads(jsonStr)

print "start"
proc = subprocess.Popen(j['script'], shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, executable='/bin/bash')
(stdout, stderr) = proc.communicate()
print stdout
print stderr
print "end"

What is wrong in my shell script which I have converted in one line? Any thoughts?

NOTE: All my other shell scripts are working which are also in one liner from the same Python subprocess call, only this is having some problem.

possible duplicate of Running bash script from within python — aruisdante
– aruisdante, Commented Mar 14, 2014 at 0:25
There's also no need to load it in like this, you can simply execute it directly. See my flagged duplicate answer. — aruisdante
– aruisdante, Commented Mar 14, 2014 at 0:26
Somehow all my other scripts are working like this way only in Python subprocess call so I don't want to change that call the way I am doing currently.. Because earlier, I tried with the way you have suggested, so some scripts were running fine and some were not.. — arsenal
– arsenal, Commented Mar 14, 2014 at 0:28

Aamir Rind · Accepted Answer · 2014-03-14 00:25:39Z

2

The error you are getting is from json.loads(jsonStr). json.loads require a str or unicode not a dict. jsonStr is already a dict you don't need json.loads

answered Mar 14, 2014 at 0:25

Aamir Rind

39.8k24 gold badges131 silver badges169 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

arsenal Over a year ago

I see.. So If I am doing like this -

proc = subprocess.Popen(jsonstr['script'], shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, executable='/bin/bash')

then it should work fine right? I tried this way and it doesn't work as well. error = string indices must be integers, not str

Aamir Rind Over a year ago

@Webby Remove this line j = json.loads(jsonStr) and try with jsonStr['script']

arsenal Over a year ago

I already tried that and this is the error I got TypeError: string indices must be integers, not str

Chris Johnson Over a year ago

Using bash syntax in a Python data structure will usually fail. Some simple constructs like strings may use a similar-enough syntax to represent the same value in both, but in general will not. For example, bash uses square brackets for purposes other than indexing an iterable (which is probably the cause of your most recent error). Either capture the bash script as a literal string (the previous triple-quote suggestion, and don't forget the leading r), or capture the script as a file. Trying to use bash syntax in Python is pointless and foolish.

Collectives™ on Stack Overflow

How to convert shell script in one line and execute it from Python subprocess?

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related