Downloading a very large XML file with PHP
<p>I currently have a script written that begins downloading an large (1.3GB) XML file from the web but I have encountered a number of problems. This is my code:</p>

<pre><code>   function readfile_chunked ($filename) { 
      $chunksize = 1*(1024*1024); 
      $buffer = ''; 
      $handle = fopen($filename, 'rb'); 
      if ($handle === false) { 
        return false; 
      } 
      while (!feof($handle)) { 
        $buffer = fread($handle, $chunksize); 
        //print $buffer; 

        $myFile = "test.xml";
        $fh = fopen($myFile, 'a') or die("can't open file");
        fwrite($fh, $buffer);
        fclose($fh);
      } 
      return fclose($handle); 
    } 
</code></pre>

<p>The first (and main) problem is the following error while downloading saying:</p>

<p><code>Fatal error: Maximum execution time of 30 seconds exceeded in /Applications/MAMP/htdocs/test/test.php on line 53</code></p>

<p>As I understand it this is basically a timeout and i've read about changing timeout settings in php.ini but i'm conscious that when this application goes live i won't be able to edit the php.ini file on the shared server.</p>

<p>This problem brings me onto my next one - i want to implement some kind of error-checking and prevention. For example, if the connection to the server goes down i'd like to be able to resume when the connection is restored. I realise this may not be possible though. An alternative would be to compare filesizes of local and remote maybe?</p>

<p>I also need to add an Accept-Encoding: gzip HTTP header in my request.</p>

<p>And that would finally bring me onto some kind of progress notification that I would like, presumably constantly polling with JavaScript comparing local and remote filesizes perhaps?</p>

<p>The first two points, however, would be the most important as currently I can't download the file I require. Any help would be appreciated.</p>
