title: Limited API - call all results from it with do while loop
body: <p>Working with this EIA API with a limit of 5000 results at once. What I'm trying to do is to loop and when I reach 5000 results loop again until all results are fetched and put in an array.</p>
  <p>This is what I have done so far but they are still 5000 on the page. Not the actual number since they are around 10k.</p>
  <p>In any case I don't know how much will be <code>length</code> when I call it but it will be more than 5000 for sure.</p>
  <pre><code>$offset = 0; 
  $length = 5000;
        
  $allData = array();
  
  $url = &quot;https://api.eia.gov/v2/international/data/v2/data?api_key=xxxxxxxxxx&amp;start=1960&amp;sort[0][direction]=desc&amp;offset=$offset&amp;length=$length&quot;;
  
  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL, $url);
  curl_setopt($ch, CURLOPT_HEADER, 0);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);  
                  
  do {
      curl_setopt($ch, CURLOPT_URL, $url);
  
      $jsonData = curl_exec($ch);
      $response = json_decode($jsonData);
                      
      foreach($response-&gt;response-&gt;data as $row1){
          $urlData[] = $row1;
      }                
      $received = $response-&gt;total; // total number of results
  
      $offset = $received - $length;
      $allData = array_merge($allData, $urlData);  
      //$offset += $received;
                  
  } while ( $offset == $received );
  
  curl_close($ch);
  </code></pre>
  <p>Update: result</p>
  <pre><code>stdClass Object
  (
      [response] =&gt; stdClass Object
          (       
              [count query execution] =&gt; 0.029705004
              [total] =&gt; 9517
              [data]
                 (
                        .....
                 )
  )
  </code></pre>
  <p>So the total should be <code>$response-&gt;response-&gt;total</code> but when I add it to the <code>while ( $response-&gt;response-&gt;total &gt; 0 );</code> it <strong>timed-out</strong></p>
  <p>UPDATE 2: <code>var_dump($response)</code></p>
  <pre><code>object(stdClass)#5003 (3) {
    [&quot;response&quot;]=&gt;
    object(stdClass)#5002 (7) {
      float(2.6497E-5)
      [&quot;count query execution&quot;]=&gt;
      float(0.037652072)
      [&quot;total&quot;]=&gt;
      int(9517)
      [&quot;data&quot;]=&gt;
      array(0) {
      }
    }
    [&quot;request&quot;]=&gt;
        [&quot;sort&quot;]=&gt;
        array(1) {
          [0]=&gt;
          object(stdClass)#1 (1) {
            [&quot;direction&quot;]=&gt;
            string(4) &quot;desc&quot;
          }
        }
        [&quot;offset&quot;]=&gt;
        int(9517)
        [&quot;length&quot;]=&gt;
        int(5000)
      }
    }
    [&quot;apiVersion&quot;]=&gt;
    string(5) &quot;2.0.2&quot;
  } 
  </code></pre>
  <p>for some reason with this setup <code>[&quot;data&quot;]=&gt; array(0) ()</code> is empty but should not be. This is actual what is (should return) returning</p>
  <pre><code>Array
  (
      [0] =&gt; stdClass Object
          (
              [count query execution] =&gt; 0.039096933
              [total] =&gt; 9517
              [data] =&gt; Array
                  (
                      [0] =&gt; stdClass Object
                          (
                              [year] =&gt; 2021
                              [Id] =&gt; 57
                              [Name] =&gt; name
                          )
  
                      [1] =&gt; stdClass Object
                          (
                              [year] =&gt; 2021
                              [Id] =&gt; 57
                              [Name] =&gt; name
                          )
                              ..........
                      [4999] =&gt; stdClass Object
                          (
                              [year] =&gt; 2021
                              [Id] =&gt; 57
                              [Name] =&gt; name
                          )
          )      
  )
  </code></pre>