python script is not running without sudo - why?

Question

This is my python script which downloads the most recent image from my S3 bucket. When I run this script using sudo python script.py it does as expected, but not when I run it as python script.py. In this case, the script finishes cleanly without exceptions or process lockup, but there is no image file.

Why does this happen? Is there something I could do at boto's library end or any other thing?

import boto
import logging


def s3_download():
    bucket_name = 'cloudadic'
    conn = boto.connect_s3('XXXXXX', 'YYYYYYY')
    bucket = conn.get_bucket(bucket_name)

    for key in bucket.list('ocr/uploads'):
        try:
            l = [(k.last_modified, k) for k in bucket]
            key = sorted(l, cmp=lambda x, y: cmp(x[0], y[0]))[-1][1]
            res = key.get_contents_to_filename(key.name)
        except:
            logging.info(key.name + ":" + "FAILED")

if __name__ == "__main__":
     s3_download()

What is "doesn't run" supposed to mean? And why do I even have to keep asking that? — Ignacio Vazquez-Abrams
– Ignacio Vazquez-Abrams, Commented Oct 14, 2016 at 10:32
Do you get an exception? Do you get a zero-length file? Does the process lock up? WHAT DOES IT MEAN? — Ignacio Vazquez-Abrams
– Ignacio Vazquez-Abrams, Commented Oct 14, 2016 at 10:36
Have you considered getting rid of the garbage exception "handling" so you can see what's actually happening? — Ignacio Vazquez-Abrams
– Ignacio Vazquez-Abrams, Commented Oct 14, 2016 at 10:38
Also, you're definitely not just running python script.py on this script since it just defines a function without calling it. Please post a minimal, complete and verifiable example. — dkasak
– dkasak, Commented Oct 14, 2016 at 10:40

Yann Vernier · Accepted Answer · 2016-10-14 10:59:57Z

1

Presumably the problem is that you try to store things somewhere your user doesn't have permission to. The second issue is that your script is hiding the error. The except block completely ignores what sort of exceptions occur (and of course consumes them so you never see them), and uses logging.info which by default doesn't show; that should probably be at least a warning and would be better if it showed something about what went wrong. By the way, odds are you shouldn't be posting S3 authentication keys here.

answered Oct 14, 2016 at 10:59

Yann Vernier

15.9k2 gold badges31 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

prashantgpt91 Over a year ago

that should not be directory permission issue, I have already done chmod -R 777 plus these are not actual S3 authentication key for exception handling I am triying, but how do I know which exception is occuring? I

Yann Vernier Over a year ago

If you don't catch the exception it will abort the script and produce a stack trace. If you want to keep running, a call to traceback.print_exc in your except block will print that same traceback for you but allow the script to continue. The logging message can be shown by either increasing severity (use warning rather than info) or configuring logging to be more verbose.

mootmoot · Accepted Answer · 2016-10-14 12:53:29Z

Always install virtual environment before you start development in python.

The issue you face is typical python newbie problem : use sudo to install all the pypi package.

When you do sudo pip install boto3, it will install pypi package to system workspace and only accessible by sudo. In such case, sudo python script.py will works, because it has the rights to access the package.

To resolve this issues and isolation of development environment (so you don't pollute different project with differnt pypi package), python developer will install python virtual environment (from above link), then use mkvirtualenv to create your project workspace, run pip install and python setup.py install to install required package to the environment, then you can run python without the sudo python.

Python virtualenv is also deployed inside production environment for the same reason.

Important Note : avoid boto and boto2. AWS no longer support them nor with bug fixing(AWS is not officially support boto2, use it at your own risk). Switch to boto3.

For the exceptional handling issues, @Yann Vernier has mentioned it. And exception error will not logged by logging.info. You may try with logging.debug or simple use raise to raise the actual exception error.

Community · Accepted Answer · 2017-05-23 12:06:56Z

0

As @Nearoo in comments had suggested to use except Exception as inst to catch exceptions.

catching the exception like this

except Exception as inst:
            print(type(inst))
            print(inst.args)
            print(inst)

Should fetch you this error if the script is compiled using python 3x If you would compile your script using python 2.7, this error wouldn't come.

Probably you might be having multiple versions of python on your system, which should be the reason for your difference in behavior of python script.py and sudo python script.py and for the same @mootmoot answer suggests to use virtualenv

'cmp' is an invalid keyword argument for this function.

Now if you would have googled this error, you should find that cmp has been deprecated in python 3x and suggestions would be to use key instead.

add these imports

import functools
from functools import cmp_to_key

and replace with these

key2 = sorted(l, key = cmp_to_key(lambda x,y: (x[0] > y[0]) - (x[0] < y[0])))[-1][1]
res = key2.get_contents_to_filename(key2.name)

(x[0] > y[0]) - (x[0] < y[0]) is an alternate to cmp(x[0], y[0])

cmp is replaced by key and cmp_to_key is used from functools lib

Check this out

edited May 23, 2017 at 12:06

CommunityBot

11 silver badge

answered Oct 14, 2016 at 13:15

prashantgpt91

1,7553 gold badges26 silver badges54 bronze badges

2 Comments

prashantgpt91 Over a year ago

Thanks man it worked. Actually I was working on openCV lib so installed Anaconda which installed different version python

mootmoot Over a year ago

@Guru : In such case, you need to install conda for Anaconda to enable your virtual environment. Because many package under conda install is not up to date, it end up no clear advatanges using conda vs virtualenv for those who not using Anaconda. kylepurdon.com/blog/…

Collectives™ on Stack Overflow

python script is not running without sudo - why?

3 Answers 3

2 Comments

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related