title: Run a single test with multiple configs in parallel using pytest
body: <p>I have to run the same test against different hosts in parallel. Currently, i'm making a bunchof pytest.main() calls. But this is not run in parallel and results are not aggregated. This is the content of runtest.py:</p>
  
  <pre><code>pytest.main('--conf=c1.txt')
  pytest.main('--conf=c2.txt')
  pytest.main('--conf=c3.txt')
  </code></pre>
  
  <p>I have just one test_host.py which looks something like:</p>
  
  <pre><code>test_pinghost(conf):
      # pings a host in config
  </code></pre>
  
  <p>Is there a way to run this in parallel  and aggregate results?
  PS:the config filejust contains the host IP</p>