What is an efficient way to parse a list of strings in Python?

Question

I am writing a script to validate a deb package is installing to a specific folder. I am pretty green and new to python. I had a script that worked with python-apt module that returned a list with file paths of all files in the package. Due to some dependency issues I can no longer use the python-apt module, so instead I am trying to call dpkg to collect the information and parse it to a list of file paths. Below is what I am using to get the list of items returned from the dpkg command. I need to parse out everything but the right of the last space. What would be the most efficient way to parse this?

self.lists = commands.getoutput("dpkg -c "+deb).split('\n')

results of this is this list:

list: ['drwxr-xr-x ejohnson/ejohnson 0 2012-03-06 15:24 ./', 'drwxr-xr-x ejohnson/ejohnson 0 2012-03-06 15:14 ./opt/', 'drwxr-xr-x ejohnson/ejohnson 0 2012-03-06 15:14 ./opt/usr/', 'drwxr-xr-x ejohnson/ejohnson 0 2012-03-06 15:15 ./opt/usr/apps/', '-r--r--r-- ejohnson/ejohnson 179491 2012-03-06 15:15 ./opt/usr/apps/xbean-spring-2.8.jar', '-rw-r--r-- ejohnson/ejohnson    518 2012-03-06 15:15 ./opt/usr/apps/Hello.class', '-r--r--r-- ejohnson/ejohnson 1901653 2012-03-06 15:15 ./opt/usr/apps/spring-1.2.6.jar']

I want to reformat the list so that each item in the list would be the item after the last space for example ['./','./opt/','./opt/usr/','./opt/usr/apps/'...]

Thanks for looking

self.lists = [line.split(' ')[-1] for line in commands.getoutput("dpkg -c "+deb).split('\n')]. Can be optimized though. — khachik
– khachik, Commented Mar 20, 2012 at 12:39

Marcin · Accepted Answer · 2012-03-20 13:21:04Z

6

Simple, just have your list in l variable and this code should work for you.

[el.split()[-1] for el in l]

edited Mar 20, 2012 at 13:21

Marcin

50.1k18 gold badges137 silver badges207 bronze badges

answered Mar 20, 2012 at 12:38

zeroos

2,2443 gold badges18 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Marcin Over a year ago

Don't suggest that anyone ever create a variable with the name of a built-in unless the intention is explicitly to shadow that built-in.

Doug Harris Over a year ago

Looks like the OP already used that name. Don't hammer zeroos for that.

zeroos Over a year ago

@Marcin, good point actually, just changed the name of this variable.

Doug Harris · Accepted Answer · 2012-03-20 12:40:06Z

2

Turn each of the strings into a list by using split. Take the last element of each list using relative list indexing.

For extra credit, do it in a one line list comprehension.

answered Mar 20, 2012 at 12:40

Doug Harris

3,4296 gold badges31 silver badges32 bronze badges

Comments

John La Rooy · Accepted Answer · 2012-03-20 20:39:26Z

2

str.rpartition may be more efficient than str.split

[x.rpartition(" ")[2] for x in your_list]

For the sample here, it is more than twice as fast

$ python -m timeit -s "L=['drwxr-xr-x ejohnson/ejohnson 0 2012-03-06 15:24 ./', 'drwxr-xr-x ejohnson/ejohnson 0 2012-03-06 15:14 ./opt/', 'drwxr-xr-x ejohnson/ejohnson 0 2012-03-06 15:14 ./opt/usr/', 'drwxr-xr-x ejohnson/ejohnson 0 2012-03-06 15:15 ./opt/usr/apps/', '-r--r--r-- ejohnson/ejohnson 179491 2012-03-06 15:15 ./opt/usr/apps/xbean-spring-2.8.jar', '-rw-r--r-- ejohnson/ejohnson    518 2012-03-06 15:15 ./opt/usr/apps/Hello.class', '-r--r--r-- ejohnson/ejohnson 1901653 2012-03-06 15:15 ./opt/usr/apps/spring-1.2.6.jar']" \
> "[x.split()[-1] for x in L]"
100000 loops, best of 3: 5.2 usec per loop

$ python -m timeit -s "L=['drwxr-xr-x ejohnson/ejohnson 0 2012-03-06 15:24 ./', 'drwxr-xr-x ejohnson/ejohnson 0 2012-03-06 15:14 ./opt/', 'drwxr-xr-x ejohnson/ejohnson 0 2012-03-06 15:14 ./opt/usr/', 'drwxr-xr-x ejohnson/ejohnson 0 2012-03-06 15:15 ./opt/usr/apps/', '-r--r--r-- ejohnson/ejohnson 179491 2012-03-06 15:15 ./opt/usr/apps/xbean-spring-2.8.jar', '-rw-r--r-- ejohnson/ejohnson    518 2012-03-06 15:15 ./opt/usr/apps/Hello.class', '-r--r--r-- ejohnson/ejohnson 1901653 2012-03-06 15:15 ./opt/usr/apps/spring-1.2.6.jar']" \
> "[x.rpartition(' ')[2] for x in L]"
100000 loops, best of 3: 2.55 usec per loop

$ python -m timeit -s "L=['drwxr-xr-x ejohnson/ejohnson 0 2012-03-06 15:24 ./', 'drwxr-xr-x ejohnson/ejohnson 0 2012-03-06 15:14 ./opt/', 'drwxr-xr-x ejohnson/ejohnson 0 2012-03-06 15:14 ./opt/usr/', 'drwxr-xr-x ejohnson/ejohnson 0 2012-03-06 15:15 ./opt/usr/apps/', '-r--r--r-- ejohnson/ejohnson 179491 2012-03-06 15:15 ./opt/usr/apps/xbean-spring-2.8.jar', '-rw-r--r-- ejohnson/ejohnson    518 2012-03-06 15:15 ./opt/usr/apps/Hello.class', '-r--r--r-- ejohnson/ejohnson 1901653 2012-03-06 15:15 ./opt/usr/apps/spring-1.2.6.jar']" \
> "[x.rsplit(' ',1)[1] for x in L]"
100000 loops, best of 3: 3.5 usec per loop

edited Mar 20, 2012 at 20:39

answered Mar 20, 2012 at 13:13

John La Rooy

306k54 gold badges378 silver badges513 bronze badges

4 Comments

Steven Rumbalski Over a year ago

x.rsplit(' ',1)[1] was fastest for me.

Niklas B. Over a year ago

@Steven: You could also use rsplit(None, 1)[1] to keep those any-amount-of-space semantics.

John La Rooy Over a year ago

@StevenRumbalski,might depend on the cpu and the version of Python amongst other things. I ran it with Python2.7.2 on a Core2 cpu

John La Rooy Over a year ago

rpartition is also fastest with Python2 and Python3 on a pentium G850

Collectives™ on Stack Overflow

What is an efficient way to parse a list of strings in Python?

3 Answers 3

3 Comments

Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related