Piping a Python script output continuosly to another program

Question

I want to output some data from a Python script continuously to another program. As an example I will use cat, this is what currently happens:

If my test1.py script is like this:

print("Hello!")

when I run ./test1.py | cat the output is Hello!, it works because the script terminates immediately after execution.

The problem occurs when I have a script that writes continuously and never terminates like test2.py:

import time

a = 0
while True:
  a += 1
  print(a)
  time.sleep(1)

Then ./test2.py | cat just hangs there because the script is not terminating.

I would like to send one number every second to cat and display it in real time, is it possible?

Padraic Cunningham · Accepted Answer · 2015-12-03 22:28:14Z

2

set flush=True in your print, your output is getting buffered, there is a nice artice Unix buffering delays output to stdout, ruins your day that explains what is going on:

import time

a = 0
while True:
  a += 1
  print(a, flush=True)
  time.sleep(1)

If you are using python2 add from __future__ import print_function

edited Dec 3, 2015 at 22:28

answered Dec 3, 2015 at 22:21

Padraic Cunningham

181k30 gold badges264 silver badges327 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

dan_s Over a year ago

This fixed my problem, I will mark this one as the correct answer because it does not require to directly use sys.stdout, it is also nice to know about the new Python3 features.

Padraic Cunningham Over a year ago

@dan_s, no worries, I added a link to an article that gives a nice explanation

PanGalactic · Accepted Answer · 2015-12-03 22:18:10Z

1

You need to flush to the stdout after printing. So your code will look like this:

import sys
import time

a = 0
while True:
    a += 1
    print(a)
    time.sleep(1)
    sys.stdout.flush()

Now running python script.py | cat will print a.

answered Dec 3, 2015 at 22:18

PanGalactic

721 silver badge8 bronze badges

Comments

Prune · Accepted Answer · 2015-12-03 22:07:28Z

0

./test2.py | tail -f

-f, --follow[={name|descriptor}]

      output  appended data as the file grows; -f, --follow, and --follow=descriptor are equivalent

answered Dec 3, 2015 at 22:07

Prune

78k14 gold badges63 silver badges83 bronze badges

Collectives™ on Stack Overflow

Piping a Python script output continuosly to another program

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related