Trying delete some files by python

Question

I am trying to delete some files that are not on the list (list_of_labeled) but the condition is never true and the condition is for some reason checked only 4010 time (it should check it 4010 * number of files times (10000))

c = 0
with open("C:\_base\MyCode\Song_genre_classification\MillionSongSubset\list_of_labeled.txt") as list_of_labeled:
    for path, subdirs, files in os.walk("C:\_base\MyCode\Song_genre_classification\MillionSongSubset\data"):
        for name in files:
            for line in list_of_labeled:
                c += 1
                if name == line[:21]:
                    # os.remove(path)
                    print(name)
print(c)

So when you step through the whole thing line-by-line in a debugger, what do you find regarding each nested loop and the expected number of iterations in each one? — Random Davis
– Random Davis, Commented Oct 14, 2019 at 16:34

Mark Tolonen · Accepted Answer · 2019-10-15 01:52:54Z

1

You open the file once outside the loop. The first time for line in list_of_labeled: is iterated, the file is exhausted. Future loops read nothing because the file is at the end. Either rewind the file each time before the for line, or load the file into a list and re-use the list.

Also this algorithm is really slow. Instead of reading the entire file and slicing the line for each name, read the file once, slice the line, and store it in a set for fast searching. Something like (untested):

with open(r"C:\_base\MyCode\Song_genre_classification\MillionSongSubset\list_of_labeled.txt") as list_of_labeled:
    lines = {line[:21] for line in list_of_labeled}

for path, subdirs, files in os.walk(r"C:\_base\MyCode\Song_genre_classification\MillionSongSubset\data"):
    for name in files:
        if name in lines:
            print(name)

edited Oct 15, 2019 at 1:52

answered Oct 14, 2019 at 16:36

Mark Tolonen

181k26 gold badges182 silver badges278 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Trying delete some files by python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related