Run shell script in python

Question

I need to execute a shell script to run the python program in via python command.

I should have to execute my python script like this

ubuntu@ip-10-32-157-231:~/hg_intcen/lib$ xvfb-run python webpage_scrapper.py  http://www.google.ca/search?q=navaspot

This script need to be executed in python program since there are huge links has to be passed to that module.

I have searched to execute this shell script in python,so i used "subprocess"

The main thing is when you run this shell command it takes some time to return the result. i need the python module to execute this command as well as it has to wait for while to return the result.This is required.

I used subprocess.Popen it doesn't return the result like what i got from the bash

import subprocess
def execute_scrapping(url):
   exe_cmd = "xvfb-run python lib/webpage_scrapper.py"+" "+str(url)
   print "cmd:"+exe_cmd
   proc = subprocess.Popen(exe_cmd,shell=True,stdin=subprocess.PIPE,stdout=subprocess.PIPE)
   time.sleep(15)
   sys.stdout.flush()
   d=proc.stdout.readlines()
   return d[1]

this above is not run into exact result. Could you please suggest me to execute the bash shell command via python and get the result?

Running Python from within Python? Ugh. Is there any reason you cannot import that script and use it as a module? — Blender
– Blender, Commented Dec 22, 2011 at 16:20
yes ofcourse. The reason is "xvfb-run".It only executes my python module.xvfb is x virtual frame buffer.along with xvfb only that module will give the result. — Nava
– Nava, Commented Dec 22, 2011 at 16:22
You might be able to replace xvfb-run with PyVirtualDisplay. Then you could import webpage_scrapper.py instead of using subprocess. — unutbu
– unutbu, Commented Dec 22, 2011 at 16:57
@unutbu It is not working with pyvirtualdisplay also..that xvfb command only doing that Is that any error? >>> from pyvirtualdisplay import Display >>> Display(visible=1, size=(320, 240)).start() >>> crawl(url) manage.py: cannot connect to X server :1188 — Nava
– Nava, Commented Dec 22, 2011 at 18:08

David K. Hess · Accepted Answer · 2011-12-22 19:13:35Z

8

Try:

proc.wait()

instead of your time.sleep(15) call.

From the docs:

Popen.wait() - Wait for child process to terminate. Set and return returncode attribute.

edited Dec 22, 2011 at 19:13

answered Dec 22, 2011 at 16:21

David K. Hess

17.3k2 gold badges53 silver badges77 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

fabmilo · Accepted Answer · 2011-12-22 18:47:00Z

2

You should use the communicate() method to wait that the external process completes.

stddata, stderr = proc.communicate()

If you have to exchange messages between the two process then look into the pexpect module:

From the website:

   import pexpect
   child = pexpect.spawn ('ftp ftp.openbsd.org')
   child.expect ('Name .*: ')
   child.sendline ('anonymous')
   child.expect ('Password:')
   child.sendline ('[email protected]')
   child.expect ('ftp> ')
   child.sendline ('cd pub')
   child.expect('ftp> ')
   child.sendline ('get ls-lR.gz')
   child.expect('ftp> ')
   child.sendline ('bye')

answered Dec 22, 2011 at 18:47

fabmilo

48.5k16 gold badges108 silver badges123 bronze badges

Collectives™ on Stack Overflow

Run shell script in python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related