title: multithreading's performance is related to global queue's length if queue is not locked?
body: <p>The requirement is: Items to deal with are stored in a global queue. Several <strong>handler thread</strong>s get item from global queue to handle. <strong>Producer thread</strong> adds item to global queue <em>continuously</em> and <em>rapidly</em>(much faster than all dealer threads' processing speed. Also, <strong>handler thread is compute-intensive</strong>. The best performance is CPU used completely).  So, I use one more <strong>countKeeping thread</strong> to keep the queue's length to a specific range, just like <em>from <strong>BOTTOM</strong> to <strong>TOP</em></strong> roughly(just to keep the memory from used too much).  </p>
  
  <p>I use <code>ManualResetEvent</code> to deal with 'can add to queue' status change. Global queue is  </p>
  
  <pre><code>Queue&lt;Object&gt; mQueue = new Queue&lt;Object&gt;;
  ManualResetEvent waitingKeeper = new ManualResetEvent(false);  
  </code></pre>
  
  <p><strong>Handler thread</strong> is  </p>
  
  <pre><code>void Handle()
  {
      while(true)
      {
          Object item;
          lock(mQueue)
          {
              if(mQueue.Count &gt; 0)
                  item = mQueue.Dequeue();
          }
          // deal with item, compute-intensive
      }
  }
  </code></pre>
  
  <p><strong>Producer thread</strong> will call AddToQueue() function to add item to mQueue.  </p>
  
  <pre><code>void AddToQueue(Object item)
  {
      waitingKeeper.WaitOne();
      lock(mQueue)
      {
          mQueue.Enqueue(item);
      }
  }
  </code></pre>
  
  <p><strong>countKeeping thread</strong> is mainly like following  </p>
  
  <pre><code>void KeepQueueingCount()
  {
      while(true)
      {
          // does not use 'lock(mQueue)'
          // because I don't need that specific count of the queue
          // I just need the queue does not cost too much memory
          if(mQueue.Count &lt; BOTTOM)
              waitingKeeper.Set();
          else if(mQueue.Count &gt; TOP)
              waitingKeeper.Reset();
          Thread.Sleep(1000);
      }
  }
  </code></pre>
  
  <p><strong>Here problom comes.</strong><br>
  When I set the BOTTOM and TOP to smaller number, like BOTTOM = 20, TOP = 100, it works well with quad core CPU(CPU utilization is high), but with single CPU it works not that well(CPU utilization fluctuates relatively great. ).<br>
  When I set the BOTTOM and TOP to larger number, like BOTTOM = 100, TOP = 300, it works well with single CPU, but not that well with quad core CPU.<br>
  Both environment, both condition, the memory is not used too much(around 50M at most).  </p>
  
  <p>Logically, larger BOTTOM and TOP will help with the performance(when memory is not used too much), bacause there are more items for <strong>handler thread</strong>s to handle. But the fact seems not like this.  </p>
  
  <p>I tried several ways to find the cause of the problem. And I just found, when I use <code>lock(mQueue)</code> in keeping thread, it works both well in two above CPU conditions.<br>
  New <strong>countKeeping thread</strong> is mainly like this  </p>
  
  <pre><code>void KeepQueueingCount()
  {
      bool canAdd = false;
      while(true)
      {
          lock(mQueue)
          {
              if(mQueue.Count &lt; BOTTOM)
                  canAdd = true;
              else if(mQueue.Count &gt; TOP)
                  canAdd = false;
          }
          if(canAdd)
              waitingKeeper.Set();
          else
              waitingKeeper.Reset();
          // I also did some change here
          // when the 'can add' status changed, will sleep longer
          // if not 'can add' status not changed, will sleep lesser
          // but this is not the main reason
          Thread.Sleep(1000);
      }
  }
  </code></pre>
  
  <p>So my questions are  </p>
  
  <ol>
  <li>When I didn't use <code>lock</code> in <strong>countKeeping thread</strong>, why the range of
  global queue affect performance(here, performance is mainly CPU
  utilization) in different CPU conditions?</li>
  <li>When I use <code>lock</code> in <strong>countKeeping thread</strong>, the performance is both
  well in different conditions. What does <code>lock</code> do really affect
  this?</li>
  <li>Is there any better way to change 'can add' status instead of using
  <code>ManualResetEvent</code>?</li>
  <li>Is there any better model fit my requirement? Or is there any better
  way to keep memory from not used too much when <strong>producer thread</strong> works
  <em>continuously</em> and <em>rapidly</em>?  </li>
  </ol>
  
  <p><strong>---UPDATE---</strong><br>
  <strong>Producer thread</strong>'s main part is as following. <em>STEP</em> is the count of items for each query from database. Querying is successively and in order until all items queryed.  </p>
  
  <pre><code>void Produce()
  {
      while(true)
      {
          // query STEP items from database
          itemList = QuerySTEPFromDB();
          if(itemList.Count == 0)
              // stop all handler thread
              // here, will wait for handler threads handle all items in queue
              // then, all handler threads exit
          else
              foreach(var item in itemList)
                  AddToQueue(item);
      }
  }
  </code></pre>