title: PHP: fopen failed "HTTP Request Failed", but response header has a status code 200
body: <p>I have a PHP script, that should connect to a proxy, chosen from a proxy list and download a file. Some of the proxies (out of 200-400 working ones) work perfectly, but others don't, and I cannot find out why.</p>
  
  <p>Here the code that connects through the proxy:</p>
  
  <pre><code>    $proxy = determine_proxy ($proxyList);
   $proxyString = 'tcp://' . $proxy['ip'] . ':' . $proxy['port'];
  
   $userAgent = $userAgents [rand (0, $agentsCount - 1)];
  
   // set up our headers
   $hdrs = array( 'http' =&gt; array(
       'method' =&gt; "GET",
       'header'=&gt; "Host: www.example.net\r\n" .
        // "User-Agent: $userAgent\r\n" .
        "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n" .
        "Accept-Language: en-us,en;q=0.5\r\n" .
        "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n" . 
        "Keep-Alive: 115\r\n" .
        "Proxy-Connection: keep-alive\r\n" .
    "Referer: http://$url",     // Setting the http-referer
       'proxy' =&gt; "$proxyString",
       'request_fulluri' =&gt; true
       )
   );
  
   echo "Using proxy: "; print_r ($proxy); echo '&lt;br&gt;';
  
   $context = stream_context_create ($hdrs);  // set up the context
  
   $timeout = 3;
   $oldTimeout = ini_set('default_socket_timeout', $timeout);
   $oldAgent = ini_set ('user_agent', $userAgent);
  
   $fp = fopen ("http://www.example.net$file", 'r', false, $context); // open the file
  
   if (!$fp) {
    echo 'fopen failed! Skipping this proxy for now...&lt;br&gt;';
    print_r ($http_response_header); echo '&lt;br /&gt;';
    unset ($http_response_header);
    flush(); @ob_flush();
    ini_set ('user_agent', $oldAgent);
    ini_set('default_socket_timeout', $oldTimeout);
    continue;
   }
  
   print_r ($http_response_header); echo '&lt;br /&gt;';
   unset ($http_response_header);
  </code></pre>
  
  <p>The bizarre thing is that the response header for the failed tries is sometimes empty, and sometimes it's the following:</p>
  
  <pre><code>   Array (
   [0] =&gt; HTTP/1.0 200 OK
   [1] =&gt; Server: falcon
   [2] =&gt; Date: Sun, 16 Jan 2011 14:06:37 GMT
   [3] =&gt; Content-Type: application/x-bittorrent
   [4] =&gt; Cache-Control: must-revalidate, post-check=0, pre-check=0
   [5] =&gt; Content-Disposition: attachment; filename="example.torrent"
   [6] =&gt; Vary: Accept-Encoding,User-Agent
   [7] =&gt; Connection: close
  )
  </code></pre>
  
  <p>And sometimes, it's this:</p>
  
  <pre><code>   Array ( 
   [0] =&gt; HTTP/1.0 200 OK 
   [1] =&gt; Server: falcon 
   [2] =&gt; Date: Sun, 16 Jan 2011 14:06:47 GMT 
   [3] =&gt; Content-Type: application/x-bittorrent 
   [4] =&gt; Cache-Control: must-revalidate, post-check=0, pre-check=0 
   [5] =&gt; Content-Disposition: attachment; filename="example2.torrent" 
   [6] =&gt; Vary: Accept-Encoding,User-Agent 
   [7] =&gt; X-Cache: MISS from proxy 
   [8] =&gt; Proxy-Connection: close
  )
  </code></pre>
  
  <p>This is a response header from a <strong><em>successful</em></strong> attempt:</p>
  
  <pre><code>HTTP/1.0 200 OK
  Server: falcon
  Date: Fri, 21 Jan 2011 18:53:00 GMT
  Content-Type: application/x-bittorrent
  Cache-Control: must-revalidate, post-check=0, pre-check=0
  Content-Disposition: attachment; filename="example3.torrent"
  Vary: Accept-Encoding,User-Agent
  X-Cache: MISS from www.example.com
  X-Cache-Lookup: MISS from www.example.com:3128
  Via: 1.0 www.example.com (squid/3.0.STABLE23-BZR)
  Proxy-Connection: close
  </code></pre>
  
  <p>I am setting the user agent to be a valid user agent string, I have checked allow_url_fopen and it is set to On.</p>
  
  <p>From RFC-2616, section 10:</p>
  
  <blockquote>
    <p>200 OK</p>
    
    <p>The request has succeeded. The
    information returned with the response
    is dependent on the method used in the
    request, for example:</p>
    
    <p>GET an entity corresponding to the
    requested resource is sent in the
    response;</p>
  </blockquote>
  
  <p>How is it possible, that the server via the proxy returns a status of 200, and still fopen fails? Does anybody have an idea about the problem and how to fix it?</p>