title: Proper use of curl_multi to make parallel asynchronous calls
body: <p>I found a <a href="https://gist.github.com/Xeoncross/2362936" rel="nofollow noreferrer">simple class</a> to do a parallel requests:</p>
  
  <pre><code>class Requests {
      public $handle;
      public function __construct() {
          $this-&gt;handle = curl_multi_init();
      }
      public function process($urls, $callback) {
          foreach($urls as $url) {
              $ch = curl_init($url);
              curl_setopt_array($ch, array(CURLOPT_RETURNTRANSFER =&gt; TRUE));
              curl_multi_add_handle($this-&gt;handle, $ch);
          }
          do {
              $mrc = curl_multi_exec($this-&gt;handle, $active);
              if ($state = curl_multi_info_read($this-&gt;handle)) {
                  $info = curl_getinfo($state['handle']);
                  $callback(curl_multi_getcontent($state['handle']), $info);
                  curl_multi_remove_handle($this-&gt;handle, $state['handle']);
              }
              usleep(10000); // stop wasting CPU cycles and rest for a couple ms
          } while ($mrc == CURLM_CALL_MULTI_PERFORM || $active);
      }
      public function __destruct() {
          curl_multi_close($this-&gt;handle);
      }
  }
  </code></pre>
  
  <p>This should be used in the following way:</p>
  
  <pre><code>$dataprocess = function($data,$info){
    echo $data;
  }
  $urls = array('url1','url2','url3');
  $rqs = new Requests();
  $rqs-&gt;process(urls,$dataprocess);
  </code></pre>
  
  <p>However it looks like not all urls are fetching (I'd estimate that only about a half of urls are fetched).</p>
  
  <p>I found <a href="http://php.net/manual/en/function.curl-multi-exec.php#80977" rel="nofollow noreferrer">this note</a> to PHP's curl_multi_exec <a href="http://php.net/manual/en/function.curl-multi-exec.php" rel="nofollow noreferrer">function description</a>:</p>
  
  <p><em>If it returns CURLM_CALL_MULTI_PERFORM you better call it again soon, as that is a signal that it still has local data to send or remote data to receive.</em></p>
  
  <p>So I suspect that this class returns too earlier or should repeat the request in some cases. But the class is controlling both <code>curl_multi_exec</code> output and <code>$active</code> parameter, so it should work fine.</p>
  
  <p>Any thoughs?</p>
  
  <p><strong>UPDATE</strong></p>
  
  <p>What I've done for the moment is that I put a cicle in the function <code>process</code> around all its code to execute it until all the URL's are retrieved (during debugging I see that after each iteration the number of unloaded URLs is reducing like 50-22-8-0).</p>
  
  <p>But I changed the class dramatically: instead of a callback function I'm passing the array with two key names (one for URL and one for content storing). So it working for me now but I still can not figure out how to do this for callback function style.</p>