title: PHP Scraping with curl - How can I debug
body: <p>I just learned what scrapping and cUrl is few hours ago, and since then I am playing with that. Nevertheless, I am facing something strange now. The here below code works fine with some sites and not with others (of course I modified the url and the xpath...). Note that I have no error raised when I test if curl_exec was executed properly. So the problem must come from somwhere after. Some my questions are as follows:  </p>
  
  <ol>
  <li>How can I check if the new DOMDocument as been created properly: if(??)  </li>
  <li>How can I check if the new DOMDocument has been populated properly with html?  </li>
  <li>...if a new DOMXPath object has been created? </li>
  </ol>
  
  <p>Hope I was clear. Thank you in advance for your replies. Cheers. Marc</p>
  
  <p>My php:</p>
  
  <pre><code>&lt;?php
  $target_url = "http://www.somesite.com";
  $userAgent = 'Googlebot/2.1 (http://www.googlebot.com/bot.html)';
  
  // make the cURL request to $target_url
  $ch = curl_init();
  curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);
  curl_setopt($ch, CURLOPT_URL,$target_url);
  curl_setopt($ch, CURLOPT_FAILONERROR, true);
  curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
  curl_setopt($ch, CURLOPT_AUTOREFERER, true);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER,true);
  curl_setopt($ch, CURLOPT_TIMEOUT, 10);
  $html= curl_exec($ch);
  
  if (!$html) {
      echo "&lt;br /&gt;cURL error number:" .curl_errno($ch);
      echo "&lt;br /&gt;cURL error:" . curl_error($ch);
      exit;
  }
  
  // parse the html into a DOMDocument
  $dom = new DOMDocument();
  @$dom-&gt;loadHTML($html);
  
  // grab all the on the page
  $xpath = new DOMXPath($dom);
  $hrefs = $xpath-&gt;query('somepath');
  
  for ($i = 0; $i &lt; $hrefs-&gt;length; $i++) {
      $href = $hrefs-&gt;item($i);
      $url = $href-&gt;getAttribute('href');
      echo "&lt;br /&gt;Link: $url";
  }
  
  ?&gt;
  </code></pre>