title: Why is my Threaded Thrift calls slow?
body: <p>My thrift definition is something like this:</p>
  
  <pre><code>list&lt;i32&gt; getValues()
  </code></pre>
  
  <p>Implemented it in C++.</p>
  
  <p>Server.cpp has the following piece of code:</p>
  
  <p>.....</p>
  
  <pre><code>std::vector&lt;int32_t&gt; store;
  
    TransferServiceHandler() {
      for(int i=0;i&lt;100000000;i++)
        store.push_back(i);
    }
  
    void getValues(std::vector&lt;int32_t&gt; &amp; _return) {
      // Your implementation goes here
  
      _return = store;
  
    }
  </code></pre>
  
  <p>.....</p>
  
  <p>Client.cpp has a simple loop in which it calls the getValues():</p>
  
  <pre><code>for(int k=0;k&lt;10;k++){
  
    clock_gettime(CLOCK_REALTIME, &amp;ds_spec);
    int64_t dstarted = ds_spec.tv_sec * 1000 + (ds_spec.tv_nsec / 1.0e6);
  
    std::vector&lt;int32_t&gt; values;
    client.getValues(values);
  
  
    clock_gettime(CLOCK_REALTIME, &amp;de_spec);
    int64_t dended = de_spec.tv_sec * 1000 + (de_spec.tv_nsec / 1.0e6);
  
    std::cout &lt;&lt; "Values size :" &lt;&lt; values.size() &lt;&lt; " in " &lt;&lt; (dended - dstarted) &lt;&lt; " ms\n";
  
  }
  </code></pre>
  
  <p>Connections are initialized and closed outside the loop.</p>
  
  <p>Usually few hundred thousand entries are returned by this call. </p>
  
  <p>When there is no data (when the lists are empty) i can see the call happening in 1ms-2ms, when i vary data there's a unpredictable delay in the transfer. Both the client and server are running on the same machine (equipped with 10Gb/s Ethernet, 8 cores and 30 GB of memory).</p>
  
  <p>How do you normally debug a situation like this? I don't think the issue is with the network since its a 10 Gigs machine and size of the data is hardly few MBs.</p>
  
  <p><a href="https://i.stack.imgur.com/ni4yh.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ni4yh.png" alt="enter image description here"></a></p>
  
  <p>I ran a benchmark with various data size and you can see the delay isn't stable for each call.</p>