Continuously watching for file change in PYTHON

Question

I have a log file which is building while a certain code is running. There exists a time step in this file that is changing and is important for my data analysis, since in each time step another file is overwritten. What I need is to have a python code which generate a few random integer for time steps let say 5 integer. I want to write a code to continuously monitor the log file and for the mentioned time step copy the overwritten file to certain directory.

Let me summarize it:

the log file contains some lines as the following 1 blablabla 2 blablabla 3 blablabla . . . 1000 blablabla

there is a file which will be overwritten in each step and I need to keep a few of them randomly. I want to save the corresponding created file of the random number in a directory while the main code is running and the log file is changing.

This is what I tried so far

def read_file():
    with open("log file", "r") as f:
        SMRF1 = f.readlines()
    return SMRF1

import os                                    #import library
import shutil                                #import library
import sys                                   #import library
import numpy as np

lower_limit = 500
upper_limit = 1000
size = 5
X1 = np.random.randint(lower_limit, upper_limit, size=(5,))
#print(type(X1))
selected_steps = np.sort(X1)
print(selected_steps)

os.mkdir("mkdir")  #make a directory 

with open('log file') as f:
    if 'random_time_step' in f.read():
        shutil.copy2("f1", "mkdir") #f1 is the file which will be overwritten in each step

but it is not monitoring the log file during the execution time. How to force it to watch for changes in log file?

Any help is highly appreciated.

if you just want to know if something was changed (and not what), take a look at os.path.getmtime() — FObersteiner
– FObersteiner, Commented Dec 2, 2019 at 16:04

Shreyanshu · Accepted Answer · 2019-12-02 16:00:47Z

1

you can use generator to monitor changes in file. checkout this example.

import time
def follow(thefile):
    thefile.seek(0,2) # Go to the end of the file
    while True:
        line = thefile.readline()
        if not line:
            time.sleep(0.1) # Sleep briefly
            continue
        yield line

answered Dec 2, 2019 at 16:00

Shreyanshu

858 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Maryam Azizi Over a year ago

Thank you for the response. I tried to modify the code with including the generator. But now it becomes an infinite loop. Could you please kindly comment on the code.

Collectives™ on Stack Overflow

Continuously watching for file change in PYTHON

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related