title: Append two pandas dataframe with different shapes and in for loop using python or pandasql
body: <p>I have two dataframe such as:</p>
  
  <p>df1:</p>
  
  <pre><code>     id  A  B  C  D
  
       1   a  b  c  d
       1   e  f  g  h
       1   i  j  k  l
  </code></pre>
  
  <p>df2:</p>
  
  <pre><code>     id  A  C  D
  
       2   x  y  z
       2   u  v  w
  </code></pre>
  
  <p>The final outcome should be:</p>
  
  <pre><code>    id  A   B  C  D
  
      1   a  b  c  d
      1   e  f  g  h
      1   i  j  k  l
      2   x     y  z
      2   u     v  w
  </code></pre>
  
  <p>These tables are generated using for loop from json files. So have to keep on appending these tables one below another. </p>
  
  <p><code>Note</code>: Two dataframes 'id' column is always different.</p>
  
  <p>My approach:</p>
  
  <p>data is a dataframe in which column 'X' has json data and has and "id" column also.</p>
  
  <pre><code>          df1=pd.DataFrame()
            for i, row1 in data.head(2).iterrows():
  
               df2= pd.io.json.json_normalize(row1["X"])
               df2.columns = df2.columns.map(lambda x: x.split(".")[-1])
               df2["id"]=[row1["id"] for i in range(df2.shape[0])]
  
  
  
               if len(df1)==0:
                  df1=df2.copy()
  
               df1=pd.concat((df1,df2), ignore_index=True)
  
  
        Error: AssertionError: Number of manager items must equal union of block items # manager items: 46, # tot_items: 49
  </code></pre>
  
  <p>How to solve this using python or pandas sql.</p>