How to create subparser with argparse from existing program in Python 3?

Question

Original post:

If one has an executable mini_program.py that uses argparse with the following structure:

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument('-X', '--attribute_matrix', type=str, help = 'Input: Path/to/Tab-separated-value.tsv')
    parser.add_argument('-y', '--target_vector', type=str, help = 'Input: Path/to/Tab-separated-value.tsv')
    opts = parser.parse_args()

if __name__ == "__main__":
    main()

How can one create a controller program parent_program.py that uses argparse (I think with subparser?) to have a similar usage to below:

python parent_program.py --help

blah-blah list of programs that can be used

then using the subprogram:

python parent_program.py mini_program --help

-X description
-y description
etc...

How could all of the parameters propagate up from mini_program.py to the parent_program.py?

EDIT (More specific with error message):

The program

import argparse
def main():
    parser = argparse.ArgumentParser()
    # Subprograms
    subprograms = parser.add_subparsers(title="subprograms")
    # ============
    # mini-program
    # ============
    parser_miniprogram = subprograms.add_parser("miniprogram")

    # Input
    parser_miniprogram.add_argument('-X', '--attribute_matrix', type=str, help = 'Input: Path/to/Tab-separated-value.tsv')
    parser_miniprogram.add_argument('-y', '--target_vector', type=str, help = 'Input: Path/to/Tab-separated-value.tsv')
    opts = parser.parse_args()
    opts_miniprogram = parser_miniprogram.parse_args()
    print(opts_miniprogram.__dict__)

if __name__ == "__main__":
    main()

Checking to make sure the docs work

# parent program
python parent_program.py --help
usage: parent_program.py [-h] {miniprogram} ...

optional arguments:
  -h, --help     show this help message and exit

subprograms:
  {miniprogram}

# miniprogram
python parent_program.py miniprogram --help
usage: parent_program.py miniprogram [-h] [-X ATTRIBUTE_MATRIX]
                                     [-y TARGET_VECTOR]

optional arguments:
  -h, --help            show this help message and exit
  -X ATTRIBUTE_MATRIX, --attribute_matrix ATTRIBUTE_MATRIX
                        Input: Path/to/Tab-separated-value.tsv
  -y TARGET_VECTOR, --target_vector TARGET_VECTOR
                        Input: Path/to/Tab-separated-value.tsv

Trying to run it:

python parent_program.py miniprogram -X ../../Data/X_iris.noise_100.tsv.gz -y ../../Data/y_iris.tsv
usage: parent_program.py miniprogram [-h] [-X ATTRIBUTE_MATRIX]
                                     [-y TARGET_VECTOR]
parent_program.py miniprogram: error: unrecognized arguments: miniprogram

Maybe something like this is useful. Or, as you mentioned, using subparsers. — pazitos10
– pazitos10, Commented Aug 30, 2018 at 18:38
@pazitos10 that git example is really interesting. I've never seen argparse used int hat way. I tried the subparser way and edited my answer. — O.rka
– O.rka, Commented Aug 30, 2018 at 19:29
You don't need to call: parser_miniprogram.parse_args(). The subparser mechanism in parser takes care of invoking the subparser with remaining arguments. — hpaulj
– hpaulj, Commented Aug 30, 2018 at 21:05

hpaulj · Accepted Answer · 2018-08-30 19:17:01Z

The parent program could have code like

import mini_program
import sys
<do its own parsing>
if 'use_mini':
    <modify sys.argv>
    mini_program.main()

As written, importing mini_program doesn't run its parser. But calling its main will, but using the list it finds in sys.argv.

The parent parser should be written in a way that it accepts arguments that it needs, and doesn't choke on inputs the mini wants, '-X' and '-y'. It would then puts those 'extra' values in a modified sys.argv, which the mini parser can handle.

parse_known_args is one way of accepting unknown arguments, https://docs.python.org/3/library/argparse.html#partial-parsing

nargs=argparse.REMAINDER, https://docs.python.org/3/library/argparse.html#nargs, is another way of collecting remaining arguments for passing on.

If mini main was written as:

def main(argv=None):
    parser = argparse.ArgumentParser()
    parser.add_argument('-X', '--attribute_matrix', type=str, help = 'Input: Path/to/Tab-separated-value.tsv')
    parser.add_argument('-y', '--target_vector', type=str, help = 'Input: Path/to/Tab-separated-value.tsv')
    opts = parser.parse_args(argv)

it could be called with

mini_program.main(['-X', 'astring','-y','another'])

that is, with an explicit argv list, instead of working through sys.argv.

Keeping the main parser from responding to a '-h' help could be tricky. subparsers is probably the cleanest way of doing that.

You could combine subparsers with the invocation of a the mini main. I won't try to work out those details now.

Another way to define the main is:

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument('-X', '--attribute_matrix', type=str, help = 'Input: Path/to/Tab-separated-value.tsv')
    parser.add_argument('-y', '--target_vector', type=str, help = 'Input: Path/to/Tab-separated-value.tsv')
    return parser

And use it as

 opts = main().parse_args()
 opts = mini_program.main().parse_args()

in other words, use main to define the parser, but delay the parsing.

Used this in combination with stackoverflow.com/questions/14071135/…

O.rka · Accepted Answer · 2018-10-03 22:58:29Z

0

My actual solution was an adaptation to the above:

# Controller
def main(argv=None):
    parser = argparse.ArgumentParser(prog="parent_program", add_help=True)
    parser.add_argument("subprogram")
    opts = parser.parse_args(argv)
    return opts.subprogram


# Initialize
if __name__ == "__main__":
    # Get the subprogram 
    subprogram = main([sys.argv[1]])
    module = importlib.import_module(subprogram)
    module.main(sys.argv[2:])

edited Oct 3, 2018 at 22:58

answered Aug 30, 2018 at 21:24

O.rka

31k78 gold badges213 silver badges336 bronze badges

Collectives™ on Stack Overflow

How to create subparser with argparse from existing program in Python 3?

Original post:

EDIT (More specific with error message):

The program

Checking to make sure the docs work

Trying to run it:

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

Original post:

EDIT (More specific with error message):

The program

Checking to make sure the docs work

Trying to run it:

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related