title: How to use vec_env or SubProcVecEnv to run multiple gyms in parallel?
body: <p>I want to use multiple (OpenAI) gyms in parallel, to speed up my RL model. The way i see it, vec_env(<a href="https://github.com/openai/baselines/blob/master/baselines/common/vec_env/vec_env.py" rel="nofollow noreferrer">github repo</a>) is the solution used at OpenAI to do exactly that. The documentation(<a href="https://stable-baselines.readthedocs.io/en/master/guide/vec_envs.html" rel="nofollow noreferrer">OpenAI documentation for baselines</a>) feels semi-useful to me until now. Specifically i have a set of questions: </p>
  
  <ol>
  <li><p>From the repo, i see that vec_env constructor is: </p>
  
  <p>def <strong>init</strong>(self, num_envs, observation_space, action_space):
  ...</p></li>
  </ol>
  
  <p>This seems weird, should it not contain a specification, of which envs i am using?</p>
  
  <ol start="2">
  <li><p>Documentation kind of confuses me even more: Seems like as user i maybe should rather use SubProcVecEnv to get multiprocessing for envs? If yes, then constructor is not clear again:</p>
  
  <p>class stable_baselines.common.vec_env.SubprocVecEnv(env_fns, start_method=None)
  ...</p></li>
  </ol>
  
  <p>env_fns is explained as: env_fns – ([Gym Environment]) Environments to run in subprocesses</p>
  
  <p>Is this: A list of strings defining the respective environments, or a list of gyms (returns from gym.make(env_name)), or something else? </p>
  
  <ol start="3">
  <li><p>If SubProcVecEnv is the way to go, how is it used: The way i see it, i just use: </p>
  
  <p>step_async(actions)
  step_wait()</p></li>
  </ol>
  
  <p>wherever i used </p>
  
  <pre><code>step(action)
  </code></pre>
  
  <p>before, just with having action of size (batch_size, action_size) instead of size (action_size).</p>
  
  <p>Can someone confirm? </p>
  
  <ol start="4">
  <li>Where from to import? I know about them from the OpenAI baselines, which are not exactly a library one would typically use, it seems to be more of a learning-tool. Is there a more 'professional' thing, where one could import them from? </li>
  </ol>