how to crawl a login protected page?
<p>I want to crawl a site search info, my script works fine for normal page, but how to crawl login protected pages? i have login info how i send this info with url?
my code:</p>

<pre><code>crawl_page("http://www.site.com/v3/search/results?start=1&amp;sortCol=MyDefault");
function crawl_page($url) {
    $html = file_get_contents($url);
    preg_match_all('~&lt;a.*?href="(.*?)".*?&gt;~', $html, $matches);
    $str = "http://www.site.com/v3/features/id?Profile=";
    foreach($matches[1] as $newurl) {
       if (strpos($newurl, $str) !== false) {
        $my_file = 'link.txt';
        $handle = fopen($my_file, 'a') or die('Cannot open file:  '.$my_file);
        $numberNewline = $newurl  . PHP_EOL;
        fwrite($handle, $numberNewline);
       }
    }
}
</code></pre>

<p>any help 
Thanks.</p>
