How to call python command line program with Jupyter cell

Question

I am using a third-party program designed to be run as a command line program, which outputs files that I later need to use in my code. I am working in Jupyter Lab and want to integrate the function calls into my code. The typical way to run this is:

python create_files.py -a input_a -b input_b -c -d

I then want to call this within my Jupyter notebook. I have been able to get it to work by using !, i.e.:

! python create_files.py -a input_a -b input_b -c -d

The problem with this is that when I want to specify input_a or input_b using variables, this doesn't work because it seems that ! expects a literal string, so to speak.

Is there a more clean way of doing this without having to alter the source code of this program (I have tried looking into that, and the code is written such that there is no simple way to call its main function.)

How about subprocess.Popen('python create_files.py -a %s -b %s -c -d' % (input_a, input_b))? — Chris
– Chris, Commented Dec 20, 2018 at 2:14

swatchai · Accepted Answer · 2018-12-20 09:04:02Z

1

On Jupyter notebook, the use of subprocess to run command line script goes like this:

Simple command line version:

 dir *.txt /s /b

On Jupyter notebook:

import subprocess
sp = subprocess.Popen(['dir', '*.txt', '/s', '/b'], \
    stderr=subprocess.PIPE, \
    stdout=subprocess.PIPE, \
    shell=True)

(std_out, std_err) = sp.communicate()   # returns (stdout, stderr)

Printing out the error message, just in case:

print('std_err: ', std_err)

Printing out the echoing message:

print('std_out: ', std_out)

I think the example is clear enough that you can adapt it to your need. Hope it helps.

answered Dec 20, 2018 at 9:04

swatchai

19k4 gold badges45 silver badges64 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

teepee Over a year ago

This is an excellent solution; thank you. There is one more feature I want to include for the output: The program I run outputs progress text which I would like to show in the Jupyter notebook in realtime as the process runs. Your code will only print out the text as a chunk after the process competes. Thanks!

William Wu · Accepted Answer · 2018-12-20 09:26:47Z

0

Your question is similar to the one:

How to execute a * .PY file from a * .IPYNB file on the Jupyter notebook?

You may use the following command, which is a little hacky:

%run -i 'create_files.py'

A "correct" way is to use the autoreload method. An example is as follows:

%load_ext autoreload
%autoreload 2
from create_files import some_function
output=some_function(input)

The reference of autoreload is as follows: https://ipython.org/ipython-doc/3/config/extensions/autoreload.html

Hope it helps.

answered Dec 20, 2018 at 9:26

William Wu

12 bronze badges

Collectives™ on Stack Overflow

How to call python command line program with Jupyter cell

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related