PHP Web scraper code causing an internal server error
<p>I'm building a (relatively) simple web scraper using PHP/CURL. This is my first time using PHP, I've tested this code in ScraperWiki and it worked just fine but I'm trying to use it on my own server and it's not running. I know the script is being read, because if I remove the simple_html_dom include I get error messages. But when it's included, I get a 500 server error.</p>

<p>Don't really know where to start trouble shooting here. Would appreciate someone looking over the code to see if there are any obvious errors? At present I just want the page to print variables on the screen so I know it's working properly, then I'm going to hook it up to mysql. So this is just in a folder on my server, along with simple_html_dom.php, and I'm accessing it by going to the www.domain.com/folder/index.php which houses the following code:</p>

<pre><code>&lt;?php
// Include simple html dom
include('simple_html_dom.php');



    // Defining the basic cURL function
    function curl($url) {
        $ch = curl_init();  // Initialising cURL
        curl_setopt($ch, CURLOPT_URL, $url);    // Setting cURL's URL option with the $url variable passed into the function
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Setting cURL's option to return the webpage data
        $data = curl_exec($ch); // Executing the cURL request and assigning the returned data to the $data variable
        curl_close($ch);    // Closing cURL
        return $data;   // Returning the data from the function
    }


$allLinks = array();
$counter = 0;

function nextPage($nextUrl){
    global $counter;
    getLinks($nextUrl);

} 

function getLinks($url){    // gets links from product list page   
    global $allLinks;
    global $counter;

    $html_content = curl($url);
    $html = str_get_html($html_content);

    foreach ($html-&gt;find("div.views-row a.imagecache-product_list") as $el) { 
        $url = $el-&gt;href . "\n";  
        $allLinks[$counter] = 'http://www.uptherestore.com';
        $allLinks[$counter] .= $url;
        $counter++;
    }

    $next = $html-&gt;find("li.pager-next a", 0); 
    if( $next != false ) $next = $next-&gt;href;

    if (isset($next)) { 
        $nextUrl = 'http://www.uptherestore.com';
        $nextUrl .= $next; 
        nextPage($nextUrl);
    }else{return;}

}

class Product{ //Creates an object class for products
    public $name = '';
    public $infoLink = '';
    public $description = '';
    public $mainImage = '';
    public $moreImages1 = '';
    public $moreImages2 = '';
    public $moreImages3 = '';
    public $moreImages4 = '';
    public $price = '';
    public $designer= '';
}


function getInfo($infoLink){    // Trawls the product pages for info  
    if(!(isset($i)))
        {$i = 0;}



    $the_content = curl($infoLink);
    $the_html = str_get_html($the_content);

    $productName = $the_html-&gt;find("#item_info h1", 0)-&gt;innertext;

        $products[$productName] = new Product;
        $products[$productName]-&gt;name = $productName;
        $products[$productName]-&gt;infoLink = $infoLink;
        $products[$productName]-&gt;designer = $the_html-&gt;find("#item_info h2", 0)-&gt;innertext;
        $products[$productName]-&gt;description = $the_html-&gt;find("#item_info .product-body", 0)-&gt;innertext; //Might cause issues because there are multiple &lt;p&gt; tags in this div
        $products[$productName]-&gt;mainImage = $the_html-&gt;find("#item_image .imagecache-product_item_default", 0)-&gt;src;

        $more1 = $the_html-&gt;find(".extra_images", 0);
        $more2 = $the_html-&gt;find(".extra_images", 1);
        $more3 = $the_html-&gt;find(".extra_images", 2);
        $more4 = $the_html-&gt;find(".extra_images", 3);

        if (isset($more1)) { 
        $products[$productName]-&gt;moreImages1 = $more1-&gt;src;
        }
if (isset($more2)) { 
        $products[$productName]-&gt;moreImages1 = $more2-&gt;src;
        }
if (isset($more3)) { 
        $products[$productName]-&gt;moreImages1 = $more3-&gt;src;
        }
if (isset($more4)) { 
        $products[$productName]-&gt;moreImages1 = $more4-&gt;src;
        }
        $products[$productName]-&gt;price = $the_html-&gt;find(".price", 0)-&gt;innertext;

// Store: $infoLink, $description, $mainImage, $moreImages, $price, $designer
echo $products[$productName]-&gt;name  . "\n";
echo $products[$productName]-&gt;description . "\n";
echo $i;
$i++;
}



getLinks("http://www.uptherestore.com/department/accessories");

foreach ($allLinks as $key =&gt; $value) {
  getInfo($value);
}

?&gt;
</code></pre>

<p>Any help would be <strong>greatly</strong> appreciated.</p>
