title: Multi-threaded C program much slower in OS X than Linux
body: <p>I wrote this for an OS class assignment that I've already completed and handed in. I posted this question yesterday, but due to "Academic Honesty" regulations I took it off until after the submission deadline. </p>
  
  <p>The object was to learn how to use critical sections. There is a <code>data</code> array with 100 monotonously increasing numbers, 0...99, and 40 threads that randomly swap two elements 2,000,000 times each. Once a second a <code>Checker</code>goes through and makes sure that there is only one of each number (which means that no parallel access happened). </p>
  
  <p>Here were the Linux times:
  </p>
  
  <pre class="lang-none prettyprint-override"><code>real    0m5.102s
  user    0m5.087s
  sys     0m0.000s
  </code></pre>
  
  <p>and the OS X times
  </p>
  
  <pre class="lang-none prettyprint-override"><code>real    6m54.139s
  user    0m41.873s
  sys     6m43.792s
  </code></pre>
  
  <p>
  I run a vagrant box with <code>ubuntu/trusty64</code> on the same machine that is running OS X. It is a quad-core i7 2.3Ghz (up to 3.2Ghz) 2012 rMBP. </p>
  
  <p>If I understand correctly, <code>sys</code> is system overhead, which I have no control over, and even then, 41s of user time suggests that perhaps the threads are running serially. </p>
  
  <p>I can post all the code if needed, but I will post the bits I think are relevant. I am using <code>pthreads</code> since that's what Linux provides, but I assumed they work on OS X. </p>
  
  <p>Creating <code>swapper</code> threads to run <code>swapManyTimes</code> routine:</p>
  
  <pre class="lang-c prettyprint-override"><code>for (int i = 0; i &lt; NUM_THREADS; i++) {
      int err = pthread_create(&amp;(threads[i]), NULL, swapManyTimes, NULL);
  }
  </code></pre>
  
  <p><code>Swapper</code> thread critical section, run in a for loop 2 million times:</p>
  
  <pre class="lang-c prettyprint-override"><code>pthread_mutex_lock(&amp;mutex);    // begin critical section
  int tmpFirst = data[first];
  data[first] = data[second];
  data[second] = tmpFirst;
  pthread_mutex_unlock(&amp;mutex);  // end critical section
  </code></pre>
  
  <p>Only one <code>Checker</code> thread is created, same way as <code>Swapper</code>. It operates by going over the <code>data</code> array and marking the index corresponding to each value with <code>true</code>. Afterwards, it checks how many indices are empty. as such:</p>
  
  <pre class="lang-c prettyprint-override"><code>pthread_mutex_lock(&amp;mutex);
  for (int i = 0; i &lt; DATA_SIZE; i++) {
      int value = data[i];
      consistency[value] = 1;
  }
  pthread_mutex_unlock(&amp;mutex); 
  </code></pre>
  
  <p>It runs once a second by calling <code>sleep(1)</code> after it runs through its <code>while(1)</code> loop. After all <code>swapper</code> threads are joined this thread is cancelled and joined as well.</p>
  
  <p>I would be happy to provide any more information that can help figure out why this sucks so much on Mac. I'm not really looking for help with code optimization, unless that's what's tripping up OS X. I've tried building it using both <code>clang</code> and <code>gcc-4.9</code> on OS X.</p>