How can I call in2csv from a Python script?

Question

When I try calling the below code, I run into the following error: "You must specify a format when providing data via STDIN (pipe)."

subprocess.call(["in2csv", "--format", "xls", a_file, ">", output_file], shell=True)

I'm not sure why this is the case because I am telling it what the initial format is. I've looked at the docs, which isn't clear about the distinction between --format and -f.

Update: I've changed it to use argparse to simplify passing the arguments following this recommendation. I'm also using Popen as used here, which is apparently safer than using shell=true flag according to the docs.

parser = argparse.ArgumentParser()
parser.add_argument('in2csv')
parser.add_argument('--format')
parser.add_argument('xls')
parser.add_argument(a_file)
parser.add_argument(">")
parser.add_argument(output_file)
args = parser.parse_args()
print args
subprocess.Popen(args)

Let cmd =["in2csv", "--format", "xls", "file.xls", ">", "file.csv"]. Then subprocess.call( ' '.join(cmd), shell=True) should work. I am not sure why subprocess.call( 'cmd, shell=True) will not work. I ran in2csv - and in2csv -- from directly the command line and received the same error: in2csv: error: You must specify a format when providing data via STDIN (pipe). — dermen
– dermen, Commented Apr 30, 2015 at 2:55
just a few suggestions subprocess.Popen can be used to pipe the output to a file using subprocess.PIPE as in eg. subprocess.Popen([cmd],stdout=fd,stderr=subprocess.PIPE,shell=False) and personally I dont think you need args parse as long as you are not passing parameters to the command line all you need is the correct string like cmd = "in2csv --format xls {0}.format(a_file)" Also please check the return from subprocess.Popen and make the call blocking to complete the command before exit — cmidi
– cmidi, Commented Apr 30, 2015 at 3:08
The code you posted is broken, and actually argparse is not what you need to answer your question. To answer your question, pass a string instead of a list of strings (like I did in the comment above). As for why the error is being thrown, I am not sure. — dermen
– dermen, Commented Apr 30, 2015 at 3:12

Will Angley · Accepted Answer · 2015-04-30 03:06:06Z

3

Errors like what you've seen are a symptom of the shell getting confused by the string passed in, for instance because of a space in a filename. It is indeed best to avoid using the shell when spawning processes from Python.

Instead of adding ">" and output_file as arguments, try redirecting the output using the stdout keyword argument, which takes a file that output will be written to.

Assuming:

a_file is a string with the name of your input file, and
output_file is a string with the name of your desired output file,

a working call might look like:

with open(output_file, 'wb') as of:
  subprocess.check_call(["in2csv", "--format", "xls", a_file],
                         stdout=of)

It's not necessary to use argparse here; it's meant for handling command lines coming in to your program, rather than going out from it.

answered Apr 30, 2015 at 3:06

Will Angley

1,3947 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How can I call in2csv from a Python script?

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related