8

There is a file (query.txt) which has some keywords/phrases which are to be matched with other files using grep. The last three lines of the following code are working perfectly but when the same command is used inside the while loop it goes into an infinite loop or something(ie doesn't respond).

import os

f=open('query.txt','r')
b=f.readline()
while b:
    cmd='grep %s my2.txt'%b    #my2 is the file in which we are looking for b
    os.system(cmd)
    b=f.readline()
f.close()

a='He is'
cmd='grep %s my2.txt'%a
os.system(cmd)
Improve this question
3
  • 1
    Why use grep when you've got the re module in Python? Commented Jan 26, 2012 at 12:34
  • This is an assignment. We are told you use grep :( Commented Jan 26, 2012 at 12:42
  • You need quotes in your constructed command line if your search pattern has a space in it, as it is in the last 3 lines of your test. You want the command to read: grep "He is" my2.txt But as it stands here, the command line is: grep He is my2.txt Commented Mar 27, 2017 at 16:25

4 Answers 4

Reset to default
10

First of all, you are not iterating over the file properly. You can simply use for b in f: without the .readline() stuff.

Then your code will blow in your face as soon as the filename contains any characters which have a special meaning in the shell. Use subprocess.call instead of os.system() and pass an argument list.

Here's a fixed version:

import os
import subprocess
with open('query.txt', 'r') as f:
    for line in f:
        line = line.rstrip() # remove trailing whitespace such as '\n'
        subprocess.call(['/bin/grep', line, 'my2.txt'])

However, you can improve your code even more by not calling grep at all. Read my2.txt to a string instead and then use the re module to perform the search. In case you do not need a regex at all, you can even simply use if line in my2_content

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Hi, Thank You for your response. And I am required to use grep because this is an assignment. And after using your code I'm getting the following error grep: is: No such file or directory
@user1171232: See my comment to eumiro's answer.
5

Your code scans the whole my2.txt file for each query in query.txt.

You want to:

  1. read all queries into a list
  2. iterate once over all lines of the text file and check each file against all queries.

Try this code:

with open('query.txt','r') as f:
    queries = [l.strip() for l in f]

with open('my2.txt','r') as f:
    for line in f:
        for query in queries:
            if query in line:
                print query, line
Improve this answer

1 Comment

Also, readline() keeps the line terminator \n in the returned string which probably confuses grep additionally.
1

This isn't actually a good way to use Python, but if you have to do something like that, then do it correctly:

from __future__ import with_statement
import subprocess

def grep_lines(filename, query_filename):
    with open(query_filename, "rb") as myfile:
        for line in myfile:
             subprocess.call(["/bin/grep", line.strip(), filename])

grep_lines("my2.txt", "query.txt")

And hope that your file doesn't contain any characters which have special meanings in regular expressions =)

Also, you might be able to do this with grep alone:

grep -f query.txt my2.txt

It works like this:

~ $ cat my2.txt 
One two
two two
two three
~ $ cat query.txt 
two two
three
~ $ python bar.py 
two two
two three
Improve this answer

7 Comments

Hi, Thank You for your response. And I am required to use grep in python because this is an assignment and we're told to do it this way. And after using your code I'm getting the following error grep: is: No such file or directory
@user1171232: What's in your query.txt? Does subprocess.call(["/bin/grep", "-e", line.strip(), filename]) work better?
Query.txt has keywords/phrases seperated by newline. ANd no the change does no good and is showing the same error.
@user1171232: I've added an example that shows the code works here. I'm not sure I can help you further without you telling me what's in your query.txt.
I wrote in the above comment what query.txt has. It has keywords seperated by newline. These are those keywords which are to be matched in my2.txt. Thanks
|
0 
$ grep -wFf query.txt my2.txt > out.txt

this will match all the keywords in query.txt with my2.txt file and save the output in out.txt

Read man grep for a description of all the possible arguments.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.