Incorporate complex bash command into python

Question

I need to call bash from python script and store result in the variable (the output is one number).

I need

output = subprocess.call('command', shell=True)

with command replaced by output from my bash one-liner which looks like this:

cat file.sam | grep -v "@" | grep "gi|519666810|ref|NM_001278619.1|"  | perl -lane '$l = 0; $F[5] =~ s/(\d+)[MX=DN]/$l+=$1/eg; print $l' | awk '{ sum+=$1} END {print sum}'

I know that simple commands work just fine:

    output = subprocess.check_output("echo 122;", shell=True)

My problem is that instead of 122 I need value from my bash one-liner. And it would be perfect if I didn't need to reformat it and could use it just as is.

Here are my attempts:

    output = subprocess.check_output("cat file.sam | grep -v "@" | grep "gi|519666810|ref|NM_001278619.1|"  | perl -lane '$l = 0; $F[5] =~ s/(\d+)[MX=DN]/$l+=$1/eg; print $l' | awk '{ sum+=$1} END {print sum}'
    ", shell=True)

      File "script.py", line 9
        output = subprocess.check_output("cat file.sam | grep -v "@" | grep "gi|519666810|ref|NM_001278619.1|"  | perl -lane '$l = 0; $F[5] =~ s/(\d+)[MX=DN]/$l+=$1/eg; print $l' | awk '{ sum+=$1} END {print sum}'
                                                                  ^
    SyntaxError: invalid syntax

Attempt two:

    output = subprocess.check_output("cat file.sam | grep -v \"@\" | grep \"gi|519666810|ref|NM_001278619.1|\"  | perl -lane '$l = 0; $F[5] =~ s/(\d+)[MX=DN]/$l+=$1/eg; print $l' | awk '{ sum+=$1} END {print sum}'
", shell=True)

  File "script.py", line 9

    output = subprocess.check_output("cat file.sam | grep -v \"@\" | grep \"gi|519666810|ref|NM_001278619.1|\"  | perl -lane '$l = 0; $F[5] =~ s/(\d+)[MX=DN]/$l+=$1/eg; print $l' | awk '{ sum+=$1} END {print sum}'
                                                                                                                                                                                                                    ^
SyntaxError: EOL while scanning string literal

I don't understand what your question is. Does it work for echo 122? What happens when you replace that with your actual command? — Blorgbeard
– Blorgbeard, Commented Mar 5, 2014 at 20:07
You need to break the bash command pipe down to individual commands for subprocess and then create the pipe with subprocess. But seriously, why not do this in python instead of messing with bash? — msvalkon
– msvalkon, Commented Mar 5, 2014 at 20:09
@Blorgbeard I edited my question to make it more clear, thanks — Perlnika
– Perlnika, Commented Mar 5, 2014 at 20:14

Perlnika · Accepted Answer · 2014-03-05 20:27:06Z

1

Thanks for help! Finally, this works:

command = "cat file.sam | grep -v \"@\" | grep \"gi|519666810|ref|NM_001278619.1|\"  | perl -lane '$l = 0; $F[5] =~ s/(\d+)[MX=DN]/$l+=$1/eg; print $l' | awk '{ sum+=$1} END {print sum}'";
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=None, shell=True)
#Launch the shell command:
output = process.communicate()[0];

answered Mar 5, 2014 at 20:27

Perlnika

5,0769 gold badges39 silver badges48 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Charles Duffy Over a year ago

I realize that this code is terse, but using subprocess with shell=True is bad practice -- it's very much the equivalent of using eval in bash, with all the same attendant security and correctness concerns. The Right Thing, long though it may be, is to specify the argv for each pipeline element in a separate Popen object.

Collectives™ on Stack Overflow

Incorporate complex bash command into python

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related