title: Parse multiple subcommands in python simultaneously or other way to group parsed arguments
body: <p>I am converting Bash shell installer utility to Python 2.7 and need to implement complex CLI so I am able to parse tens of parameters (potentially up to ~150). These are names of Puppet class variables in addition to a dozen of generic deployment options, which where available in shell version. </p>
  
  <p>However after I have started to add more variables I faced are several challenges:
  1. I need to group parameters into separate dictionaries so deployment options are separated from Puppet variables. If they are thrown into the same bucket, then I will have to write some logic to sort them, potentially renaming parameters and then dictionary merges will not be trivial.
  2. There might be variables with the same name but belonging to different Puppet class, so I thought subcommands would allow me to filter what goes where and avoiding name collisions. </p>
  
  <p>At the momment I have implemented parameter parsing via simply adding multiple parsers:</p>
  
  <pre><code>parser = argparse.ArgumentParser(description='deployment parameters.')
  env_select = parser.add_argument_group(None, 'Environment selection')
  env_select.add_argument('-c', '--client_id',  help='Client name to use.')
  env_select.add_argument('-e', '--environment', help='Environment name to use.')
  setup_type = parser.add_argument_group(None, 'What kind of setup should be done:')
  setup_type.add_argument('-i', '--install', choices=ANSWERS, metavar='', action=StoreBool, help='Yy/Nn Do normal install and configuration')
  # MORE setup options
  ...
  args, unk = parser.parse_known_args()
  config['deploy_cfg'].update(args.__dict__)
  
  pup_class1_parser = argparse.ArgumentParser(description=None)
  pup_class1 = pup_class1_parser.add_argument_group(None, 'Puppet variables')
  pup_class1.add_argument('--ad_domain', help='AD/LDAP domain name.')
  pup_class1.add_argument('--ad_host', help='AD/LDAP server name.')
  # Rest of the parameters
  
  args, unk = pup_class1_parser.parse_known_args()
  config['pup_class1'] = dict({})
  config['pup_class1'].update(args.__dict__)
  # Same for class2, class3 and so on.
  </code></pre>
  
  <p>The problem with this approach that it does not solve issue 2. Also first parser consumes "-h" option and rest of parameters are not shown in help. </p>
  
  <p>I have tried to use <a href="https://stackoverflow.com/questions/10448200/how-to-parse-multiple-sub-commands-using-python-argparse">example selected as an answer</a> but I was not able to use both commands at once. </p>
  
  <pre><code>## This function takes the 'extra' attribute from global namespace and re-parses it to create separate namespaces for all other chained commands.
  def parse_extra (parser, namespace):
    namespaces = []
    extra = namespace.extra
    while extra:
      n = parser.parse_args(extra)
      extra = n.extra
      namespaces.append(n)
  
    return namespaces
  
  pp = pprint.PrettyPrinter(indent=4)
  
  argparser=argparse.ArgumentParser()
  subparsers = argparser.add_subparsers(help='sub-command help', dest='subparser_name')
  
  parser_a = subparsers.add_parser('command_a', help = "command_a help")
  ## Setup options for parser_a
  parser_a.add_argument('--opt_a1', help='option a1')
  parser_a.add_argument('--opt_a2', help='option a2')
  
  parser_b = subparsers.add_parser('command_b', help = "command_b help")
  ## Setup options for parser_a
  parser_b.add_argument('--opt_b1', help='option b1')
  parser_b.add_argument('--opt_b2', help='option b2')
  
  
  ## Add nargs="*" for zero or more other commands
  argparser.add_argument('extra', nargs = "*", help = 'Other commands')
  
  namespace = argparser.parse_args()
  pp.pprint(namespace)
  extra_namespaces = parse_extra( argparser, namespace )
  pp.pprint(extra_namespaces)
  </code></pre>
  
  <p>Results me in:</p>
  
  <pre><code>$ python argtest.py command_b --opt_b1 b1 --opt_b2 b2 command_a --opt_a1 a1
  usage: argtest.py [-h] {command_a,command_b} ... [extra [extra ...]]
  argtest.py: error: unrecognized arguments: command_a --opt_a1 a1
  </code></pre>
  
  <p>The same result was when I tried to define parent with two child parsers.</p>
  
  <p><strong>QUESTIONS</strong></p>
  
  <ol>
  <li>Can I somehow use parser.add_argument_group for argument parsing or is it just for the grouping in help print out? It would solve issue 1 without missing help side effect. Passing it as <code>parse_known_args(namespace=argument_group)</code> (if I correctly recall my experiments) gets all the variables (thats ok) but also gets all Python object stuff in resulting dict (that's bad for hieradata YAML)</li>
  <li>What I am missing in the second example to allow to use multiple subcommands? Or is that impossible with argparse?</li>
  <li>Any other suggestion to group command line variables? I have looked at Click, but did not find any advantages over standard argparse for my task. </li>
  </ol>
  
  <p>Note: I am sysadmin, not a programmer so be gently on me for the non object style coding. :)</p>
  
  <p>Thank you</p>
  
  <p><strong>RESOLVED</strong> 
  Argument grouping solved via the answer suggested by <a href="https://stackoverflow.com/users/901925/hpaulj">hpaulj</a>. </p>
  
  <pre><code>import argparse
  import pprint
  parser = argparse.ArgumentParser()
  
  group_list = ['group1', 'group2']
  
  group1 = parser.add_argument_group('group1')
  group1.add_argument('--test11', help="test11")
  group1.add_argument('--test12', help="test12")
  
  group2 = parser.add_argument_group('group2')
  group2.add_argument('--test21', help="test21")
  group2.add_argument('--test22', help="test22")
  
  args = parser.parse_args()
  pp = pprint.PrettyPrinter(indent=4)
  
  d = dict({})
  
  for group in parser._action_groups:
      if group.title in group_list:
          d[group.title]={a.dest:getattr(args,a.dest,None) for a in group._group_actions}
  
  print "Parsed arguments"
  pp.pprint(d)
  </code></pre>
  
  <p>This gets me desired result for the issue No.1. until I will have multiple parameters with the same name. Solution may look ugly, but at least it works as expected.</p>
  
  <pre><code>python argtest4.py --test22 aa  --test11 yy11 --test21 aaa21
  Parsed arguments
  {   'group1': {   'test11': 'yy11', 'test12': None},
      'group2': {   'test21': 'aaa21', 'test22': 'aa'}}
  </code></pre>