python script is not executed via crontab

Question

I have a python script test.py (fyi, it contains some crawling logic) and want it to be executed every one minute.

So I write a shell script weekjob.sh( tried each all below)

python /path/test.py
/usr/bin/python /path/test.py
cd /path && /usr/bin/python test.py

and then register a job

* * * * * /bin/sh /path/weekjob.sh

But python is not executed.

When I execute the shell script like sh weekjob.sh it worked just fine. And when I changed the contents of weekjob.sh as mkdir something cronjob also worked well, too.

what is wrong with my python cron job?

(OS is centOS)

please look at the crond logs: unix.stackexchange.com/questions/207/… — Ozgur Vatansever
– Ozgur Vatansever, Commented Jul 24, 2015 at 5:28
@ozgur I did, but it showed just fine success log like CROND[22259] (user) CMD (/usr/bin/python /path/weekMobileJob.py) — Soeun Park
– Soeun Park, Commented Jul 24, 2015 at 5:34

Soeun Park · Accepted Answer · 2015-07-27 05:23:32Z

2

I found the answer. It was the problem of proxy thing. cronlog shows below error.

Traceback (most recent call last):
  File "/home/deploy/crawlingScript/proxyTest.py", line 41, in <module>
    converter = xml2csv(new_url, username, password, output_file_name, encoding="utf-8")
  File "/home/deploy/crawlingScript/converterOld.py", line 13, in __init__
    response = urllib2.urlopen(request)
  File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib64/python2.6/urllib2.py", line 391, in open
    response = self._open(req, data)
  File "/usr/lib64/python2.6/urllib2.py", line 409, in _open
'_open', req)
  File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain
result = func(*args)
  File "/usr/lib64/python2.6/urllib2.py", line 1190, in http_open
return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib64/python2.6/urllib2.py", line 1165, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [Errno 110] Connection timed out>

I set proxy in .bashrc but cron has some problem with it I think.

So, I just added some code for proxy in .py file.

and found another solution thanks to @WillemM

* * * * * . /path/.bashrc; /bin/sh /path/to/sh/test.sh

.bashrc is the file that contains proxy setting.

edited Jul 27, 2015 at 5:23

answered Jul 27, 2015 at 2:33

Soeun Park

3494 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

wmoco_6725 Over a year ago

The crontab executed script does not have the same environment as a script that you run on the command line.

Soeun Park Over a year ago

@WillemM Ah.. I should have find more about crontab..! Thank you, M.

wmoco_6725 · Accepted Answer · 2015-07-24 06:30:16Z

-1

Instead of doing

* * * * * /bin/sh /path/weekjob.sh

why dont' you drop the /bin/sh like this:

* * * * * /path/weekjob.sh

And first ensure that that script is executable:

chmod +x /path/weekjob.sh

answered Jul 24, 2015 at 6:30

wmoco_6725

3,2191 gold badge14 silver badges12 bronze badges

4 Comments

Soeun Park Over a year ago

I did it already.. and shell script is actually keep running. I checked it on /var/log/cron

tripleee Over a year ago

While this is a reasonable suggestion, it does nothing to diagnose or solve the actual problem in the question. Furthrtmire, the shell script which solely calls the Python script is also completely superfluous.

wmoco_6725 Over a year ago

Please share the content of your python script, I think we may need to look in there for a diagnosis.

Soeun Park Over a year ago

@WillemM and as you said, python script has some flaw. Python script is about crawling data from some website which needs authentication. It was not the problem of authentication though cause I set id and pwd to authenticate.

Collectives™ on Stack Overflow

python script is not executed via crontab

2 Answers 2

2 Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related