Setting a timeout during a loop in Python?

Question

So I'm running a loop through a directory where an external script, run by subprocess.Popen, is moving through the directory and performing calculations on each file. The external script is a bit unstable and occasionally freezes when it runs into files that it doesn't know how to handle. Is there a way to add a timeout function to subprocess.Popen so that I can skip over that file and move on to the next?

EDIT: Here's my loop:

def automate():
    os.chdir("/home/mlts/dir")
    working_dir = b"/home/mts/dir"
    for filename in os.listdir(working_dir):
        if filename.endswith(".MTS"):
            try:
                print("Performing calculations on {filename!r}...".format(**vars()))
                try:
                    os.remove("mts_tbl.txt")
                except OSError:
                    pass
                time.sleep(3)
                p = Popen(["run_command.command", "-f", "a"], cwd=working_dir, stdin=PIPE)
                p.communicate(input=b"\n".join([b"1", str(filename), b"8", b"alloy-liquid", b"0", b"x", b"5", b"4", b"-1.6", b"4", b"1", b"0"]))

One can help you over timeout part as is but for skip over the file script freezes on, you must provide code — MohitC
– MohitC, Commented Oct 17, 2015 at 22:55
To do 2 things at once, 1=run the function, 2=cancel after a certain amount of time has passed, you use Multiprocessing or Threading. Personally, I would call the function as a multiprocessing process, and terminate the process after the allowed time has passed. Doug Hellmann's page is at pymotw.com/2/multiprocessing/basics.html#terminating-processes — user4171906
– user4171906, Commented Oct 17, 2015 at 23:53
Would this approach allow for the continuation of the loop onto the next file in the directory? — Scott
– Scott, Commented Oct 18, 2015 at 4:39

jfs · Accepted Answer · 2015-10-18 12:46:20Z

To force timeout on Python 2 (where .communicate(timeout=) doesn't exist in stdlib), you could use threading.Timer():

p = Popen(...)
t = Timer(3, p.kill) # kill in 3 seconds
t.start()
p.communicate(...)
t.cancel() # no need to kill, the process is dead already

The complete example:

import os
import traceback
from glob import glob
from subprocess import Popen, PIPE
from threading import Timer

os.chdir("/home/mls/dir") #XXX assume mlts is a typo in the question
for filename in glob(b"*.MTS"):
    print("Performing calculations on {filename!r}...".format(**vars()))
    try:
        os.remove("mts_tbl.txt")
    except OSError:
        pass # ignore
    try:
        p = Popen(["run_command.command", "-f", "a"], stdin=PIPE) #XXX no cwd
        t = Timer(3, p.kill)
        t.start()
        p.communicate(input=b"\n".join([b"1", filename,
                b"8\nalloy-liquid\n0\nx\n5\n4\n-1.6\n4\n1\n0"]))
        t.cancel()
    except Exception:
        traceback.print_exc()

This code assumes that you want to run the child process in the current working directory of the parent Python script.

On Python 3 or if subprocess32 is installed; you could pass timeout parameter to .communicate() method instead of using Timer().

Collectives™ on Stack Overflow

Setting a timeout during a loop in Python?

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related