title: Benchmarking Newtonsoft.Json deserialization: from stream and from string
body: <p>I'm interested in performance (speed, memory usage) comparison of two approaches how to deserialize HTTP response JSON payload using <a href="https://www.newtonsoft.com/json" rel="noreferrer">Newtonsoft.Json</a>.</p>
  
  <p>I'm aware of <a href="https://www.newtonsoft.com/json/help/html/Performance.htm" rel="noreferrer">Newtonsoft.Json's Performance Tips</a> to use streams, but I wanted to know more and have hard numbers. I've written simple benchmark using <a href="https://benchmarkdotnet.org/" rel="noreferrer">BenchmarkDotNet</a>, but I'm bit puzzled by results (see numbers below).</p>
  
  <p>What I got:</p>
  
  <ul>
  <li>parsing from stream is always faster, but not really much</li>
  <li>parsing small and "medium" JSON has better or equal memory usage when using string as input</li>
  <li>significant difference in memory usage starts to be seen with large JSON (where string itself ends up in LOH)</li>
  </ul>
  
  <p>I didn't have time to do proper profiling (yet), I'm bit surprised by memory overhead with stream approach (if there's no error). Whole code is <a href="https://github.com/wdolek/allocator" rel="noreferrer">here</a>.</p>
  
  <h2>?</h2>
  
  <ul>
  <li>Is my approach correct? (usage of <code>MemoryStream</code>; simulating <code>HttpResponseMessage</code> and its content; ...)</li>
  <li>Is there any issue with benchmarking code?</li>
  <li>Why do I see such results?</li>
  </ul>
  
  <h2>Benchmark setup</h2>
  
  <p>I'm preparing <code>MemoryStream</code> to be used over and over within benchmark run:</p>
  
  <pre><code>[GlobalSetup]
  public void GlobalSetup()
  {
      var resourceName = _resourceMapping[typeof(T)];
      using (var resourceStream = Assembly.GetExecutingAssembly().GetManifestResourceStream(resourceName))
      {
          _memory = new MemoryStream();
          resourceStream.CopyTo(_memory);
      }
  
      _iterationRepeats = _repeatMapping[typeof(T)];
  }
  </code></pre>
  
  <h2>Stream deserialization</h2>
  
  <pre><code>[Benchmark(Description = "Stream d13n")]
  public async Task DeserializeStream()
  {
      for (var i = 0; i &lt; _iterationRepeats; i++)
      {
          var response = BuildResponse(_memory);
  
          using (var streamReader = BuildNonClosingStreamReader(await response.Content.ReadAsStreamAsync()))
          using (var jsonReader = new JsonTextReader(streamReader))
          {
              _serializer.Deserialize&lt;T&gt;(jsonReader);
          }
      }
  }
  </code></pre>
  
  <h2>String deserialization</h2>
  
  <p>We first read JSON from stream to string, and then run deserialization - another string is being allocated, and after that used for deserialization.</p>
  
  <pre><code>[Benchmark(Description = "String d13n")]
  public async Task DeserializeString()
  {
      for (var i = 0; i &lt; _iterationRepeats; i++)
      {
          var response = BuildResponse(_memory);
  
          var content = await response.Content.ReadAsStringAsync();
          JsonConvert.DeserializeObject&lt;T&gt;(content);
      }
  }
  </code></pre>
  
  <h2>Common methods</h2>
  
  <pre><code>private static HttpResponseMessage BuildResponse(Stream stream)
  {
      stream.Seek(0, SeekOrigin.Begin);
  
      var content = new StreamContent(stream);
      content.Headers.ContentType = new MediaTypeHeaderValue("application/json");
  
      return new HttpResponseMessage(HttpStatusCode.OK)
      {
          Content = content
      };
  }
  
  [MethodImpl(MethodImplOptions.AggressiveInlining)]
  private static StreamReader BuildNonClosingStreamReader(Stream inputStream) =&gt;
      new StreamReader(
          stream: inputStream,
          encoding: Encoding.UTF8,
          detectEncodingFromByteOrderMarks: true,
          bufferSize: 1024,
          leaveOpen: true);
  
  </code></pre>
  
  <h2>Results</h2>
  
  <h3>Small JSON</h3>
  
  <p>Repeated 10000 times</p>
  
  <ul>
  <li>Stream: mean 25.69 ms, 61.34 MB allocated</li>
  <li>String: mean 31.22 ms, 36.01 MB allocated</li>
  </ul>
  
  <h3>Medium JSON</h3>
  
  <p>Repeated 1000 times</p>
  
  <ul>
  <li>Stream: mean 24.07 ms, 12 MB allocated</li>
  <li>String: mean 25.09 ms, 12.85 MB allocated</li>
  </ul>
  
  <h3>Large JSON</h3>
  
  <p>Repeated 100 times</p>
  
  <ul>
  <li>Stream: mean 229.6 ms, 47.54 MB allocated, objects got to Gen 1</li>
  <li>String: mean 240.8 ms, 92.42 MB allocated, objects got to Gen 2!</li>
  </ul>
  
  <hr>
  
  <h2>Update</h2>
  
  <p>I went trough source of <code>JsonConvert</code> and found out that it internally uses <code>JsonTextReader</code> with <code>StringReader</code> when deserializing from <code>string</code>: <a href="https://github.com/JamesNK/Newtonsoft.Json/blob/master/Src/Newtonsoft.Json/JsonConvert.cs#L816" rel="noreferrer">JsonConvert:816</a>. Stream is involved there as well (of course!).</p>
  
  <p>Then I decided to dig more into <code>StreamReader</code> itself and I was stunned at first sight - it is always allocating array buffer (<code>byte[]</code>): <a href="https://github.com/Microsoft/referencesource/blob/master/mscorlib/system/io/streamreader.cs#L244" rel="noreferrer">StreamReader:244</a>, which explains its memory use.</p>
  
  <p>This gives me answer to "why". Solution is simple - use smaller buffer size when instantiating <code>StreamReader</code> - minimum buffer size defaults to 128 (see <code>StreamReader.MinBufferSize</code>), but you can supply any value <code>&gt; 0</code> (check one of ctor overload).</p>
  
  <p>Of course buffer size <em>has</em> effect on processing data. Answering what buffer size I should then use: <em>it depends</em>. When expecting smaller JSON responses, I think it is safe to stick with small buffer.</p>