read from subprocess output python

Question

I am running a subprocess using 'Popen'. I need to block till this subprocess finishes and then read its output.

p = Popen(command, stdin=PIPE, stdout=PIPE, stderr=PIPE, encoding="utf-8")
p.communicate():
output = p.stdout.readline()
print(output)

I get an error that

ValueError: I/O operation on closed file.

How can I read the output after the subprocess finishes, I do not want to use poll() though as the subprocess takes time and I would need to wait for its completion anyway.

Any particular reason you're not using subprocess.run() or the legacy subprocess.check_output()? You should avoid Popen if you can precisely because it's tricky to get right. — tripleee
– tripleee, Commented Aug 2, 2018 at 18:29
@jasonharper, p.communicate() returns bound method Popen.communicate of <subprocess.Popen object at 0x105e84e10, not the output of the subprocess code — marc
– marc, Commented Aug 2, 2018 at 18:34
output, error = p.communicate() should work, that output looks like you are just printing p.communicate (without the parentheses). — cdarke
– cdarke, Commented Aug 2, 2018 at 18:36

Niko Fohr · Accepted Answer · 2023-07-01 15:13:15Z

2

This should work:

p = Popen(command, stdin=PIPE, stdout=PIPE, stderr=PIPE, encoding="utf-8")
output, error = p.communicate()

print(output)
if error:
    print('error:', error, file=sys.stderr)

However, subprocess.run() is preferred these days:

p = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

print("output:", p.stdout)

if proc.stderr:
    print("error:", p.stderr, file=sys.stderr)

edited Jul 1, 2023 at 15:13

Niko Fohr

35.3k12 gold badges113 silver badges117 bronze badges

answered Aug 2, 2018 at 18:44

cdarke

44.8k8 gold badges85 silver badges86 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Zags · Accepted Answer · 2018-08-02 19:26:09Z

0

Use subprocess.check_output. It returns the output of the command.

answered Aug 2, 2018 at 19:26

Zags

41.9k16 gold badges125 silver badges157 bronze badges

Comments

wiesiu_p · Accepted Answer · 2024-04-30 19:09:23Z

actually when I wanted to run python3 -c "import time;time.sleep(3);print(1)" with subprocess.run or subprocess.Popen I faced that problem of not collecting outputs and returning before child process ended. The only feasible solution was/is for me asyncio code (for 3.6) below but for newer version will be pretty similar:

class ShellCommand(Command):
  OK = 0
  ENCODING = 'utf-8'

  def run(self, args: List[str]) -> str:
    cmd = ' '.join(args)
    logging.info('executing %s', cmd)
    async def _run(cmd: str, future: asyncio.Future):
        proc = await asyncio.create_subprocess_shell(
            cmd,
            stdout=asyncio.subprocess.PIPE,
            stderr=asyncio.subprocess.STDOUT)
        stdout, stderr = await proc.communicate()
        if proc.returncode != type(self).OK:
          raise ValueError(f'Return code: {proc.returncode}', f'Message: {stderr.decode(type(self).ENCODING)}')
        future.set_result(stdout.decode(type(self).ENCODING))
    asyncio.set_event_loop(asyncio.new_event_loop())
    loop = asyncio.get_event_loop()
    future = asyncio.Future()
    asyncio.ensure_future(_run(cmd, future), loop=loop)
    loop.run_until_complete(future)
    loop.close()
    return future.result()

Collectives™ on Stack Overflow

read from subprocess output python

3 Answers 3

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related