1

I'm not too familiar with python, can anyone tell me how I can open files using variables in python? I want to automate this in a script to run the same task across multiple directories

Here machine and inputfile are variables.. I tried the following code but keep getting errors

file = open(machine + '/' + inputfile)

printing works fine.. i.e. the variables were populated correctly:

print 'Input file is "', inputfile -> abc 
print 'Machine "', machine -> xyz

Hence file location should be ./xyz/abc

Error: The error I get is file ./machine/inputfile does not exist, i.e. instead of taking the value of the variable machine and inputfile it is taking them as is.

Apologies if this is too trivial a question

4
  • 1
    That concept will work, though you should look into using os.path.join (and possibly with as a context manager for the file), Commented Sep 30, 2014 at 9:23
  • 1
    your file is already open. read this and work with file. Commented Sep 30, 2014 at 9:25
  • What errors do you receive? Maybe machine or inputfile are not strings, but ints or some other type, in which case you should convert it to string using str(machine) or str(inputfile) before concatenating them with +. Commented Sep 30, 2014 at 9:27
  • @HasanRamezani- apologies i think i might have written it such that it seemed as if I was reading machine and inputfile variables from the file, I meant to say that I checked that these variables are already populated.. edited the question to reflect that. Commented Sep 30, 2014 at 14:20

2 Answers 2

1

In a general sense, there is nothing wrong with your code. There is probably something wrong with your path and/or file name.

This is how I would do it (on windows)

import os

dir='C:/Users/xxx' # You can use forward slashes on Windows
file='somefile.txt'

full_path=os.path.join(dir,file) # OS independent way of building paths

with open(full_path,'r') as f: # 'with' will automatically close file for you,
    for line in f: # Do something with the file
        print line
Sign up to request clarification or add additional context in comments.

Comments

1

First, I don't recommend naming your variable file because file() is a built-in alias to the open() function in Python. You can do this. But I recommend against it as many Python programmers will cringe when they see it.

Perhaps "in_filename" for the input filename and "inputfile" for the file() object.

More to the point once you have the open file you have to call methods on it (explicitly or implicitly) in order to use the contents of the file.

There are several ways to do this. (As others have pointed out you should use os.path.join() to portably combine a path to a base filename):

#!python
import os
path = '/some/path'
in_filename = 'somefile.txt'
inputfile = open(os.path.join(path, in_filename), 'r')
data = inputfile.read()  # fetch all contents in one string
inputfile.seek(0) # reset file object point to beginning of file
data_lines = inputfile.readlines()  # fetch all contents as list of lines
inputfile.seek(0) # rest file object again
while True:
    line = inputfile.readline() # one line at a time (line is a string)
    if not line:
        break   # length of line is zero at end of file
        # Note: empty strings are "False" in Python; so this detects EOF
inputfile.close()  # close inputfile 

# alternatively using "context management" features
with open(os.path.join(path, in_filename), 'r') as infile:
    for line in infile:
        do_something(line)

# infile is implicitly closed on exit from the context management "suite"
# (indented block of code)

In this sample I'm showing how to slurp in the entire file as a single string; how to slurp it in as a list of lines, how to iterate over it line-by-line and how to use the "context manager" (with statement) to iterate over it line-by-line).

Notice that iterating over a file is implicitly calling the .readline() (singular) method. (That's actually orthogonal to the with context management we used to open it. with only guarantees that the file will be automatically closed after the end of our indented block of code ... even if some exception were raised by the open or any other operations we preformed on it. for ... in ...: creates and iterator and uses it; and many objects in Python define iteration semantics. For files that's based on calls to .readline())

I'm also showing how to .seek() back to the beginning of the file (which you might not do often but in this case it's a way to show all these alternative ways of reading the file without having to repeatedly .close() and re-open it).

Note that I'm explicitly use the access option in my open() call to declare that my file handler will be read-only. If I wanted to read from and write to the file I'd use 'w+' and so on).

I'm not showing the option to the .read() method which allows you to specify the amount of data you wish to read for each call. For normal text files this is generally not very useful because then you'd have to handle any partial lines yourself which is rather messy code. If you were to attempt to read some very large file using my examples then you might cause your system to page/swap and suffer some drastic performance and responsiveness issues with your system. (On a typical, modern, laptop you'll be fine so long as the file is less than a gigabyte or two in size).

Using .read(xxx) (with specific numbers of characters to be read for any call) can be useful if you have a file containing fixed length records (text or binary). However, it's rare these days for anyone to manipulation fixed length file structures using Python or other scripting languages. If you were dealing with non-text file (such as manipulating the headers of MP3 files, or images, etc, then you might use such methods (and you'd likely want to also use the struct module to "pack" data from its textual representation into the specific machine-native representation required by your binary file formats).

1 Comment

Thanks for the detailed answer. The file reading part seems to be working fine. In particular I guess I was looking for os.path.join(), since putting variables directly in the open() call seems to use them as is instead of using their values.. I'll check out the os.path.join and comment back as soon as the error is fixed :)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.