title: recursive web crawler perl
body: <p>I am trying to write a minimal web crawler. The aim is to discover new URLs from the seed and crawl these new URLs further. The code is as follows:</p>
  
  <pre><code>use strict;
  use warnings;
  use Carp;
  use Data::Dumper;
  use WWW::Mechanize;
  
  my $url = "http://foobar.com"; # example
  my %links;
  
  my $mech = WWW::Mechanize-&gt;new(autocheck =&gt; 1);
  $mech-&gt;get($url);
  my @cr_fronteir = $mech-&gt;find_all_links();
  
  foreach my $links (@cr_fronteir) {
      if ( $links-&gt;[0] =~ m/^http/xms ) {
          $links{$links-&gt;[0]} = $links-&gt;[1];
      }
  }
  </code></pre>
  
  <p>I am stuck here, how could I proceed further to crawl the links in %links and also, how do I add depth to prevent overflow. Suggestion are appreciated.</p>