title: Index not found in a dataframe
body: <p>I'm trying to solve a reinforcement learning problem with a simple q learning algorithme.
  I must add a new state to my table if the agent don't already know it.
  The probleme I have is that when I learn the new state, I have the folowing error: <code>'None of [x] are in the [index]'</code></p>
  
  <p>I did a little test code to understand what's happening and solve the problems when I encounter one but I don't find solution to this one.</p>
  
  <p>Here's my code:</p>
  
  <pre><code>import pandas as pd
  import numpy as np
  import random
  
  actions = [0, 1]
  obs = (0)
  
  
  q_table = pd.DataFrame(columns=actions)
  
  
  def check_state_exist(state):
      global q_table
      if state not in q_table.index:
          # append new state to q table
          q_table = q_table.append(
              pd.Series(
                  [0]*len(actions),
                  index=q_table.columns,
                  name=state,
              )
          )
  def choose_action(state):
      global actions
      check_state_exist(state)
      # action selection
      if np.random.uniform() &lt; 0.9:
          # choose best action
          state_action = q_table.loc[state, :]
          state_action = state_action.reindex(np.random.permutation(state_action.index))     # some actions have same value
          action = np.argmax(state_action)
      else:
          # choose random action
          action = np.random.choice(actions)
      return action
  
  
  for i in range(50):
  
      rand = random.randrange(100)
      if rand == 1:
          obs = 'rare'
      else: 
          obs = [rand, random.randrange(10)]
          obs = tuple(obs)
  
      choose_action(obs)
  </code></pre>
  
  <p>'None of [(56, 5)] are in the [index]'</p>
  
  <pre><code>print(q_table)
  
           0  1
  (56, 5)  0  0
  </code></pre>
  
  <p>Thanks for your help</p>