title: Display OpenAI gym in Jupyter notebook only
body: <p>I want to play with the OpenAI gyms in a notebook, with the gym being rendered inline.</p>
  
  <p>Here's a basic example:</p>
  
  <pre><code>import matplotlib.pyplot as plt
  import gym
  from IPython import display
  %matplotlib inline
  
  env = gym.make('CartPole-v0')
  env.reset()
  
  for i in range(25):
     plt.imshow(env.render(mode='rgb_array'))
     display.display(plt.gcf())    
     display.clear_output(wait=True)
     env.step(env.action_space.sample()) # take a random action
  
  env.close()
  </code></pre>
  
  <p>This works, and I get see the gym in the notebook:</p>
  
  <p><a href="https://i.stack.imgur.com/h0Qy0b.png" rel="noreferrer"><img src="https://i.stack.imgur.com/h0Qy0b.png" alt="gym in notebook"></a></p>
  
  <p><strong>But!</strong> it also opens an interactive window that shows precisely the same thing. <strong>I don't want this window to be open:</strong></p>
  
  <p><a href="https://i.stack.imgur.com/YKWkC.png" rel="noreferrer"><img src="https://i.stack.imgur.com/YKWkC.png" alt="interactive window"></a></p>