Run a complex grep command with Python

Question

Im trying to run this grep compound command that works fine on cmd

grep Rec sy.log | grep e612r2246s768 | grep 2013-07 | grep -oh "'.*'" | wc -c

But something is wrong here, and I can't see it yet:

import commands
commands.getstatus("""/bin/grep Rec /var/log/sy.log | /bin/grep e612r2246s768 | /bin/grep 2013-07 | /bin/grep -oh "'.*'" | /usr/bin/wc -c""")
Out[2]: 'ls: cannot access /bin/grep Rec /var/log/sy.log | /bin/grep e612r2246s768 | /bin/grep 2013-07 | /bin/grep -oh "\'.*\'" | /usr/bin/wc -c: No such file or directory'

Using subprocess:

import subprocess
cmd = ["""/bin/grep Rec /var/log/sy.log | /bin/grep e612r2246s768 | /bin/grep 2013-07 | /bin/grep -oh "'.*'" | /usr/bin/wc -c"""]
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE)
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
/home/www-data/gpslistener/scripts/<ipython-input-24-0881e54c5eab> in <module>()
----> 1 proc = subprocess.Popen(cmd, stdout=subprocess.PIPE)

/usr/lib/python2.7/subprocess.pyc in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags)
    677                             p2cread, p2cwrite,
    678                             c2pread, c2pwrite,
--> 679                             errread, errwrite)
    680 
    681         if mswindows:

/usr/lib/python2.7/subprocess.pyc in _execute_child(self, args, executable, preexec_fn, close_fds, cwd, env, universal_newlines, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite)
   1247                     if fd is not None:
   1248                         os.close(fd)
-> 1249                 raise child_exception
   1250 
   1251 

OSError: [Errno 2] No such file or directory

PD: The paths are ok, thanks

"Deprecated since version 2.6: The commands module has been removed in Python 3. Use the subprocess module instead." — user1907906
– user1907906, Commented Nov 15, 2013 at 15:19

Goku · Accepted Answer · 2013-11-15 15:44:48Z

11

Ok, this way worked to me:

>>>import subprocess
>>>cmd = ["""/bin/grep 'Rec' /var/log/sy.log | /bin/grep e612r2246s768 | /bin/grep 2013-07 | /bin/grep -oh "'.*'" | /usr/bin/wc -c"""]
>>>print subprocess.check_output(cmd,shell=True)
>>>365829

answered Nov 15, 2013 at 15:44

Goku

1,8506 gold badges25 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Ashish Karpe Over a year ago

can you please explain why 3 times """ in cmd ??

Zei Over a year ago

@AshishKarpe It's a bit late, but for anyone wondering, it makes it so that you can have quotes of any type inside the string. e.g. """hello"'"world""" would output hello"'"world.

iruvar · Accepted Answer · 2013-11-15 15:45:23Z

8

With the usual caveat related to shell injection, the easiest way to execute a shell pipeline is by passing in shell=True

cmd = r'''/bin/grep Rec /var/log/syslog | \
    /bin/grep e612r2246s768 |\
    /bin/grep 2013-07 |\
/bin/grep -oh "'.*'" |\
/usr/bin/wc -c'''
subprocess.check_output(cmd, shell=True)

answered Nov 15, 2013 at 15:45

iruvar

23.5k7 gold badges58 silver badges83 bronze badges

Collectives™ on Stack Overflow

Run a complex grep command with Python

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related