error when trying to regex python3 subprocess command output

Question

I've been banging my head against a wall here...

I want to run a particular cli command from within a python script and then regex it's output. But I get the following error:

"File "speedtestcork.py", line 18, in regex_speedtest
    for line in data.split("\n"):
TypeError: Type str doesn't support the buffer API"

Here's my short code:

import time
import re
import speedtest_cli
import subprocess



def regex_speedtest(data):
        #output, errors = data.communicate()
        #output1 = str(output)
        Down_Regex = re.compile(r'(Download:).(\d+\.\d+).(Mbit/s)')
        Up_Regex = re.compile(r'(Upload:).(\d+\.\d+).(Mbit/s)')

        for line in data.split("\n"):          # code seems to break at this line.
                print(line)
                #Downmatch = Down_Regex.search(line)
                #Upmatch = Up_Regex.search(line)
        #return (Downmatch.group(2), Upmatch.group(2))



while True:
        now = time.strftime("%H:%M:%S %d %m %Y")
        process = subprocess.Popen(["speedtest-cli", "--server", "2432"], shell=True, stdout=subprocess.PIPE)
        output, errors = process.communicate()
        down, up = regex_speedtest(output)

This should be simple to debug but I cannot find the solution.

Used the following post but still cannot resolve:

Regex woes with Subprocess output

Thanks in advance,

Paul

Sergei Lebedev · Accepted Answer · 2016-03-11 11:37:02Z

2

In Python3 Popen.communicate returns bytes, while regex_speedtest expects str input.

Here's an updated version:

def regex_speedtest(data):
    Down_Regex = re.compile(rb'(Download:).(\d+\.\d+).(Mbit/s)')
    Up_Regex = re.compile(rb'(Upload:).(\d+\.\d+).(Mbit/s)')

    for line in data.splitlines():   
        print(line)

I've changed the r prefix to rb, which transforms string literals into byte literals and replaced split("\n") call with splitlines, since this works for both str and bytes.

Update: as noted by Duncan in the comments, replacing "\n" with b"\n" in the split call would work just as well.

edited Mar 11, 2016 at 11:37

answered Mar 11, 2016 at 11:25

Sergei Lebedev

2,67920 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Duncan Over a year ago

or you could use data.split(b'\n') as the problem was simply the mixing of str and bytes in the split call.

Sergei Lebedev Over a year ago

The regex matches would've failed as well, try re.compile("foo").search(b"foo").

Duncan Over a year ago

Yes, I wasn't complaining about that part of your answer, just that you implied that .split() wouldn't work for bytes when in fact it does. (Not that there is anything wrong with using .splitlines() either).

Tullio_IRL Over a year ago

Thanks guys, my first ever post here, well impressed with the response time and knowledge :) Solving this has highlighted another issue: Traceback (most recent call last): File "speedtestcork.py", line 37, in <module> down, up = regex_speedtest(output) TypeError: 'NoneType' object is not iterable

Sergei Lebedev Over a year ago

This might indicate that you don't return anything from regex_speedtest, which makes the unpacking fail.

|

Collectives™ on Stack Overflow

error when trying to regex python3 subprocess command output

1 Answer 1

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related