title: Performance issues with multithreading and stdout
body: <p>I'm working on a benchmark application which uses an user-defined number of threads to do the processing. I'm also working on a visualizer application for the benchmark results.</p>
  
  <p>The benchmark itself is written in C++ (and uses pthreads for the threading) while the visualizer is written in Python.</p>
  
  <p>Right now, what I'm doing to make the two talk is piping <code>stdout</code> from the benchmark to the visualizer. This has the advantage of being able to use a tool like <code>netcat</code> to run the benchmark on one machine and the visualizer on another.</p>
  
  <p><img src="https://i.stack.imgur.com/ldKww.png" alt="High level diagram of the application"></p>
  
  <p>A bit about the benchmark:</p>
  
  <ul>
  <li>It is very CPU bound</li>
  <li>Each thread writes important data (i.e. data that I need for the visualizer) every few 10's of milliseconds.</li>
  <li>Each datum printed is a line of 5 to 20 characters.</li>
  <li>As stated previously, the number of threads is highly variable (can be 1, 2, 40, etc.)</li>
  <li>Even though it is important that the data isn't mangled (e.g. that one thread preempts another during a <code>printf</code>/<code>cout</code>, causing the printed data to be interleaved with the output on another thread), it's not very important that the writes are done in the correct order. </li>
  </ul>
  
  <p>Example regarding the last point:</p>
  
  <pre><code>// Thread 1 prints "I'm one\n" at the 3 seconds mark
  // thread 2 prints "I'm two\n" at the 4 seconds mark
  
  // This is fine
  I'm two
  I'm one
  
  // This is not
  I'm I'm one
   two
  </code></pre>
  
  <p>On the benchmark, I switched from <code>std::cout</code> to <code>printf</code> due to it being closer to a <code>write</code> (2) in order to minimize the chance of interleaving between outputs of different threads.</p>
  
  <p>I'm worried that writing to <code>stdout</code> from multiple threads will be cause for a bottleneck as the number of threads increases. It is quite important that the output-for-visualization part of the benchmark is extremely light on resources so as not to skew the results.</p>
  
  <p>I'm looking for ideas on an efficient way of making my two applications talk without impacting the performance of my benchmark more than the absolute essential. Any ideas? Any of you tackled problems like this before? Any smarter/cleaner solutions?</p>