title: Split nested JSON File into two JSONs according to their ID?
body: <p>I have nested JSON File which loaded as python dictionary called <code>movies_data</code> as follows:</p>
  <pre><code>with open('project_folder/data_movie_absa.json') as infile:
    movies_data = json.load(infile)
  </code></pre>
  <p>It has the following structure:</p>
  <pre><code>{ &quot;review_1&quot;: {&quot;tokens&quot;: [&quot;Best&quot;, &quot;show&quot;, &quot;ever&quot;, &quot;!&quot;], 
                 &quot;movie_user_4&quot;: {&quot;aspects&quot;: [&quot;O&quot;, &quot;B_A&quot;, &quot;O&quot;, &quot;O&quot;], &quot;sentiments&quot;: [&quot;B_S&quot;, &quot;O&quot;, &quot;O&quot;, &quot;O&quot;]},  
                 &quot;movie_user_6&quot;: {&quot;aspects&quot;: [&quot;O&quot;, &quot;B_A&quot;, &quot;O&quot;, &quot;O&quot;], &quot;sentiments&quot;: [&quot;B_S&quot;, &quot;O&quot;, &quot;O&quot;, &quot;O&quot;]}}, 
  
    &quot;review_2&quot;: {&quot;tokens&quot;: [&quot;Its&quot;, &quot;a&quot;, &quot;great&quot;, &quot;show&quot;], 
                 &quot;movie_user_1&quot;: {&quot;aspects&quot;: [&quot;O&quot;, &quot;O&quot;, &quot;O&quot;, &quot;B_A&quot;], &quot;sentiments&quot;: [&quot;O&quot;, &quot;O&quot;, &quot;B_S&quot;, &quot;O&quot;]}, 
                 &quot;movie_user_6&quot;: {&quot;aspects&quot;: [&quot;O&quot;, &quot;O&quot;, &quot;O&quot;, &quot;B_A&quot;], &quot;sentiments&quot;: [&quot;O&quot;, &quot;O&quot;, &quot;B_S&quot;, &quot;O&quot;]}},
  
    &quot;review_3&quot;: {&quot;tokens&quot;: [&quot;I&quot;, &quot;love&quot;, &quot;this&quot;, &quot;actor&quot;, &quot;!&quot;],  
                 &quot;movie_user_17&quot;: {&quot;aspects&quot;: [&quot;O&quot;, &quot;O&quot;, &quot;O&quot;, &quot;B_A&quot;, &quot;O&quot;], &quot;sentiments&quot;: [&quot;O&quot;, &quot;B_S&quot;, &quot;O&quot;, &quot;O&quot;, &quot;O&quot;]}, 
                 &quot;movie_user_23&quot;: {&quot;aspects&quot;: [&quot;O&quot;, &quot;O&quot;, &quot;O&quot;, &quot;B_A&quot;, &quot;O&quot;], &quot;sentiments&quot;: [&quot;O&quot;, &quot;B_S&quot;, &quot;O&quot;, &quot;O&quot;, &quot;O&quot;]}},
  
    &quot;review_4&quot;: {&quot;tokens&quot;: [&quot;Bad&quot;, &quot;movie&quot;], 
                 &quot;movie_user_1&quot;: {&quot;aspects&quot;: [&quot;O&quot;, &quot;B_A&quot;], &quot;sentiments&quot;: [&quot;B_S&quot;, &quot;O&quot;]}, 
                 &quot;movie_user_6&quot;: {&quot;aspects&quot;: [&quot;O&quot;, &quot;B_A&quot;], &quot;sentiments&quot;: [&quot;B_S&quot;, &quot;O&quot;]}}
  
  ...
  }
  </code></pre>
  <p>It has 3324 key-value pairs (i.e., up to key review_3224). I want to split this file into two json files (<code>train_movies.json</code>, <code>test_movies.json</code>) based on specific list of keys:</p>
  <pre><code>test_IDS = ['review_2', 'review_4']
  
  with open(&quot;train_movies.json&quot;, &quot;w&quot;, encoding=&quot;utf-8-sig&quot;) as outfile_train, open(&quot;test_movies.json&quot;, &quot;w&quot;, encoding=&quot;utf-8-sig&quot;) as outfile_test:
    for review_id, review in movies_data.items():
      if review_id in test_IDS:
        outfile = outfile_test
        outfile.write('{&quot;%s&quot;: &quot;%s&quot;}' % (review_id, movies_data[review_id]))
        
      else:
        outfile = outfile_train
        outfile.write('{&quot;%s&quot;: &quot;%s&quot;}' % (review_id, movies_data[review_id]))
    outfile.close()
  </code></pre>
  <p>for test_movies.json I have the following structure:</p>
  <pre><code>{&quot;review_2&quot;: &quot;{'tokens': ['Its', 'a', 'great', 'show'], 
              'movie_user_4': {'aspects': ['O', 'O', 'O', 'B_A'], 'sentiments': ['O', 'O', 'B_S', 'O']}, 
              'movie_user_6': {'aspects': ['O', 'O', 'O', 'B_A'], 'sentiments': ['O', 'O', 'B_S', 'O']}}&quot;}
  
  {&quot;review_4&quot;: &quot;{'tokens': ['Bad', 'movie'], 
                 'movie_user_1': {'aspects': ['O', 'B_A'], 'sentiments': ['B_S', 'O']},
                 'movie_user_6': {'aspects': ['O', 'B_A'], 'sentiments': ['B_S', 'O']}}&quot;}
  </code></pre>
  <p>Unfortunately, this structure has some <strong>problems</strong> like inconsistent double <strong>quotes</strong> (<code>&quot; vs. '</code>), <strong>no commas</strong> between reviews etc... Therefore, I have the following problem by reading <code>test_movies.json</code> as <code>json</code> file:</p>
  <pre><code>with open('project_folder/test_movies.json') as infile:
    testing_data = json.load(infile)
  </code></pre>
  <p>The error message:</p>
  <pre><code>
  JSONDecodeError                           Traceback (most recent call last)
  &lt;ipython-input-10-3548a718f421&gt; in &lt;module&gt;()
        1 with open('/content/gdrive/My Drive/project_folder/test_movies.json') as infile:
  ----&gt; 2   testing_data = json.load(infile)
  
  1 frames
  /usr/lib/python3.6/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
      342         if s.startswith('\ufeff'):
      343             raise JSONDecodeError(&quot;Unexpected UTF-8 BOM (decode using utf-8-sig)&quot;,
  --&gt; 344                                   s, 0)
      345     else:
      346         if not isinstance(s, (bytes, bytearray)):
  
  JSONDecodeError: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)
  </code></pre>
  <p>The <strong>desired output</strong> should have a correct json structure like the original <code>movies_data</code> so python could read it correctly as a dict.</p>
  <p>Could you help me please to correct my python code?</p>
  <p>Thank you in advance!</p>