title: Saving Pandas DataFrame and meta-data to JSON format
body: <p>I have a need to save a Pandas DataFrame, along with some metadata to a file in JSON format.  (The JSON format is a requirement.)  </p>
  
  <p><strong>Background</strong><br>
  <strong>A)</strong> I can successfully read/write my rather large Pandas Dataframe from/to JSON using <code>DataFrame.to_json()</code> and <code>DataFrame.from_json()</code>.  No problems.</p>
  
  <p><strong>B)</strong> I have no problems saving my metadata (dict) to JSON using <code>json.dump()</code>/<code>json.load()</code></p>
  
  <p><br>
  <strong>My first attempt</strong><br>
  Since Pandas does not support DataFrame metadata directly, my first thought was to</p>
  
  <pre><code>top_level_dict = {}
  top_level_dict['data'] = df.to_dict()
  top_level_dict['metadata'] = {'some':'stuff'}
  json.dump(top_level_dict, fp)
  </code></pre>
  
  <p><br>
  <strong>Failure modes</strong><br>
  <strong>C)</strong>   I have found that even the simplified case of    </p>
  
  <pre><code>df_dict = df.to_dict()
  json.dump(df_dict, fp)
  </code></pre>
  
  <p>fails with:</p>
  
  <pre><code>TypeError: key (u'US', 112, 5, 80, 'wl') is not a string
  </code></pre>
  
  <p><strong>D)</strong> Investigating, I've found that the complement also fails.</p>
  
  <pre><code>df.to_json(fp)
  json.load(fp)
  </code></pre>
  
  <p>fails with</p>
  
  <pre><code>384             raise ValueError("No JSON object could be decoded")
  ValueError: Expecting : delimiter: line 1 column 17 (char 16)
  </code></pre>
  
  <p>So it appears that Pandas JSON format and the Python's JSON library are not compatible.</p>
  
  <p>My first thought is to chase down a way to modify the <code>df.to_dict()</code> output of <strong>C</strong> to make it amenable to Python's JSON library, but I keep hearing "If you're struggling to do something in Python, you're probably doing it wrong." in my head.</p>
  
  <p><br>
  <strong>Question</strong><br>
  What is the cannonical/recommended method for adding metadata to a Pandas DataFrame and storing to a JSON-formatted file? </p>
  
  <p>Python 2.7.10<br>
  Pandas 0.17</p>
  
  <p><strong>Edit 1:</strong><br>
  While trying out Evan Wright's great answer, I found the source of my problems: Pandas (as of 0.17) does not like saving Multi-Indexed DataFrames to JSON.  The library I had created to save my (Multi-Indexed) DataFrames is quietly performing a <code>df.reset_index()</code> before calling <code>DataFrame.to_json()</code>.  My newer code was not.  So it was <code>DataFrame.to_json()</code> burping on the MultiIndex.  </p>
  
  <p>Lesson: Read the documentation kids, even when it's <em>your own</em> documentation.</p>
  
  <p><strong>Edit 2:</strong>  </p>
  
  <p>If you need to store both the DataFrame and the metadata in a <em>single</em> JSON object, see my answer below.</p>