python multiple inputs and multiple outputs

Question

I have written a script in python, which works on a single file. I couldn't find an answer to make it run on multiple files and to give output for each file separately.

out = open('/home/directory/a.out','w')
infile = open('/home/directory/a.sam','r')

for line in infile:
    if not line.startswith('@'):
        samlist = line.strip().split()
        if 'I' or 'D' in samlist[5]:
            match = re.findall(r'(\d+)I', samlist[5]) # remember to chang I and D here aswell
            intlist = [int(x) for x in match]
##            if len(intlist) < 10:
            for indel in intlist:
                if indel >= 10:
##                    print indel
            ###intlist contains lengths of insertions in for each read
            #print intlist
                    read_aln_start = int(samlist[3])
                    indel_positions = []
                    for num1, i_or_d, num2, m in re.findall('(\d+)([ID])(\d+)?([A-Za-z])?', samlist[5]):
                        if num1:
                            read_aln_start += int(num1)
                        if num2:
                            read_aln_start += int(num2)
                        indel_positions.append(read_aln_start)
                #print indel_positions
                    out.write(str(read_aln_start)+'\t'+str(i_or_d) + '\t'+str(samlist[2])+ '\t' + str(indel) +'\n')
out.close()

I would like my script to take multiple files with names like a.sam, b.sam, c.sam and for each file give me the output : aout.sam, bout.sam, cout.sam

Can you please pass me either a solution or a hint.

Regards, Irek

Have you tried wrapping that script in a function and passing the names of the input and output file as parameters? — tobias_k
– tobias_k, Commented Jul 18, 2013 at 9:29
if 'I' or 'D' in samlist[5] doesn't do what you think it does. This condition is always true. — Sven Marnach
– Sven Marnach, Commented Jul 18, 2013 at 9:31
I don't think it's always true. Only some lines contain I or D. Most of them is actually without any of these letters, so then the condition is false. — Irek
– Irek, Commented Jul 18, 2013 at 9:38
@Irek Python interprets any non-empty string as a boolean True, so the above condition is essentially if bool('I') or ('D' in samlist[5]): — tobias_k
– tobias_k, Commented Jul 18, 2013 at 9:40
And the correct way of writing that code is if samlist[5] in ('I', 'D') — Jon Clements
– Jon Clements, Commented Jul 18, 2013 at 9:41

falsetru · Accepted Answer · 2013-07-18 09:45:13Z

4

Loop over filenames.

input_filenames = ['a.sam', 'b.sam', 'c.sam']
output_filenames = ['aout.sam', 'bout.sam', 'cout.sam']
for infn, outfn in zip(input_filenames, output_filenames):
    out = open('/home/directory/{}'.format(outfn), 'w')
    infile = open('/home/directory/{}'.format(infn), 'r')
    ...

UPDATE

Following code generate output_filenames from given input_filenames.

import os

def get_output_filename(fn):
    filename, ext = os.path.splitext(fn)
    return filename + 'out' + ext

input_filenames = ['a.sam', 'b.sam', 'c.sam'] # or glob.glob('*.sam')
output_filenames = map(get_output_filename, input_filenames)

edited Jul 18, 2013 at 9:45

answered Jul 18, 2013 at 9:30

falsetru

371k69 gold badges769 silver badges659 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

Irek Over a year ago

Not exactly what I'm looking for. I still need to write all of the filenames. It's cool until you have 100 files in the directory

falsetru Over a year ago

@Irek, Added another code that generate output_filenames from input_filenames.

Irek Over a year ago

Ok great. Is it possible to also generate fileinput names?

falsetru Over a year ago

@Irek, you can use glob.glob('*.sam'). But that will only work for first time. Once the script run, glob.glob('*.sam') will also include output filenames because both input filenames, output filenames ends with .sam.

Irek Over a year ago

Ok. That's not a problem, I put my output to separate directory, that should do the trick

|

tobias_k · Accepted Answer · 2013-07-18 12:08:10Z

1

I'd recommend wrapping that script in a function, using the def keyword, and passing the names of the input and output files as parameters to that function.

def do_stuff_with_files(infile, outfile):
    out = open(infile,'w')
    infile = open(outfile,'r')
    # the rest of your script

Now you can call this function for any combination of input and output file names.

do_stuff_with_files('/home/directory/a.sam', '/home/directory/a.out')

If you want to do this for all files in a certain directory, use the glob library. To generate the output filenames, just replace the last three characters ("sam") with "out".

import glob
indir, outdir = '/home/directory/', '/home/directory/out/'
files = glob.glob1(indir, '*.sam')
infiles  = [indir  + f              for f in files]
outfiles = [outdir + f[:-3] + "out" for f in files]
for infile, outfile in zip(infiles, outfiles):
    do_stuff_with_files(infile, outfile)

edited Jul 18, 2013 at 12:08

answered Jul 18, 2013 at 9:36

tobias_k

83.1k12 gold badges130 silver badges186 bronze badges

5 Comments

falsetru Over a year ago

glob.glob('/home/directory/*.out') will not work, because you have to create output file before run the script.

tobias_k Over a year ago

@falsetru Yes, realized that, too. Borrowed your method for that. ;-)

Irek Over a year ago

@tobias_k if I would like also to make my outfiles in different directory?

user91 Over a year ago

How can I modify the code if I want to make process on my inFile before writing the outFile? @tobias_k

tobias_k Over a year ago

@user91 What do you mean? You just add that code to do_stuff_with_files

tlewis3348 · Accepted Answer · 2013-07-18 12:55:53Z

The following script allows working with an input and output file. It will loop over all files in the given directory with the ".sam" extension, perform the specified operation on them, and output the results to a separate file.

Import os
# Define the directory containing the files you are working with
path = '/home/directory'
# Get all the files in that directory with the desired
# extension (in this case ".sam")
files = [f for f in os.listdir(path) if f.endswith('.sam')]
# Loop over the files with that extension
for file in files:
    # Open the input file
    with open(path + '/' + file, 'r') as infile:
        # Open the output file
        with open(path + '/' + file.split('.')[0] + 'out.' +
                               file.split('.')[1], 'a') as outfile:
            # Loop over the lines in the input file
            for line in infile:
                # If a line in the input file can be characterized in a
                # certain way, write a different line to the output file.
                # Otherwise write the original line (from the input file)
                # to the output file
                if line.startswith('Something'):
                    outfile.write('A different kind of something')
                else:
                    outfile.write(line)
    # Note the absence of either a infile.close() or an outfile.close()
    # statement. The with-statement handles that for you

Collectives™ on Stack Overflow

python multiple inputs and multiple outputs

3 Answers 3

9 Comments

5 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

9 Comments

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related