title: Perl Mechanize - how to make a script running faster with less overhead
body: <p>Problem: I have a list of 2500 websites and need to grab a thumbnail screenshot of them. How do I do that?
  I could try to parse the sites either with Perl.- Mechanize would be a good thing.
  Note: i only need the results as a thumbnails that are a maximum 240 pixels in the long dimension. At the moment i have a solution which is slow and does not give back thumbnails: How to make the script running faster with less overhead - spiting out the thumbnails</p>
  
  <p><strong>Prerequisites:</strong> 
  addon/mozrepl/
  the module WWW::Mechanize::Firefox;
  the module imager</p>
  
  <p><strong>First Approach:</strong> Here is a first Perl solution:</p>
  
  <pre><code> use WWW::Mechanize::Firefox;
   my $mech = WWW::Mechanize::Firefox-&gt;new();
   $mech-&gt;get('http://google.com');
   my $png = $mech-&gt;content_as_png();
  </code></pre>
  
  <p><strong>Outline:</strong> This returns the given tab or the current page rendered as PNG image.
  All parameters are optional. $tab defaults to the current tab. If the coordinates are given,that rectangle will be cut out. The coordinates should be a hash with the four usual entries, left,top,width,height.This is specific to WWW::Mechanize::Firefox.</p>
  
  <p>As i understand from the perldoc that option with the coordinates, it is not the resize of the whole page it's just a rectangle cut out of it.... well  the WWW::Mechanize::Firefox takes care for how to save screenshots. Well i forgot to mention that i only need to have the images as small thumbnails - so we do not have to have a very very large files...i only need to grab a thumbnail screenshot of them. I have done a lookup on cpan for some module that scales down the $png  and i found out Imager </p>
  
  <p>The mecha-module does not concern itself with resizing images. Here we have the various image modules on CPAN, like Imager.
  Imager - Perl extension for Generating 24 bit Images:  Imager is a module for creating and altering images. It can read and write various image formats, draw primitive shapes like lines,and polygons, blend multiple images together in various ways, scale, 
  crop, render text and more. I installed the module - but i did not have extended my basic-approach </p>
  
  <p>What i have tried allready; here it is: </p>
  
  <pre><code>#!/usr/bin/perl
  
  use strict;
  use warnings;
  use WWW::Mechanize::Firefox;
  
  my $mech = new WWW::Mechanize::Firefox();
  
  open(INPUT, "&lt;urls.txt") or die $!;
  
  while (&lt;INPUT&gt;) {
          chomp;
          print "$_\n";
          $mech-&gt;get($_);
          my $png = $mech-&gt;content_as_png();
          my $name = "$_";
          $name =~s/^www\.//;
          $name .= ".png";
          open(OUTPUT, "&gt;$name");
          print OUTPUT $png;
          sleep (5);
  }
  </code></pre>
  
  <p>Well this does not care about the size: </p>
  
  <p>See the output commandline: </p>
  
  <pre><code>linux-vi17:/home/martin/perl # perl mecha_test_1.pl
     www.google.com
      www.cnn.com
      www.msnbc.com
  command timed-out at /usr/lib/perl5/site_perl/5.12.3/MozRepl/Client.pm line 186
  linux-vi17:/home/martin/perl # 
  </code></pre>
  
  <p>This is my source ... see a snippet [example]of the sites i have in the url-list.</p>
  
  <p>urls.txt [the list of sources ]</p>
  
  <pre><code>www.google.com
  www.cnn.com
  www.msnbc.com
  news.bbc.co.uk
  www.bing.com
  www.yahoo.com
  </code></pre>
  
  <p><strong>Question:</strong> how to extend the solution either to make sure that it does not stop in a  time out. and - it does only store little thumbnails 
  Note:again: i only need the results as a thumbnails that are a maximum 240 pixels in the long dimension. As a prerequisites,  i allready have installed the module imager </p>
  
  <p>How to make the script running faster with less overhead - spiting out the thumbnails</p>
  
  <p>Love to hear from you! greetings
  zero</p>
  
  <p><strong>Update:</strong> in addition to Schwerms idea which are very  very intersting i found a <a href="http://www.perlmonks.org/?node_id=901572" rel="nofollow">intersting Monkthread </a>which talks about the same timeouts: </p>
  
  <p>Is there a way to specify Net::Telnet timeout with WWW::Mechanize::Firefox? At the moment my internet connection is very slow and sometimes I get error with</p>
  
  <pre><code> $mech-&gt;get(): command timed-out at /usr/local/share/perl/5.10.1/MozRepl/Client.pm line 186
  </code></pre>
  
  <p>Perhaps i have to loook after the mozrepl-Timeout-configuration!? But after all: This is weird and I don't know where that timeout comes from. Maybe it really is Firefox timing out as it is busy synchronously fetching some result. As you see in the trace, WWW::Mechanize::Firefox polls every second (or so) to see whether Firefox has fetched a page.</p>
  
  <p>If it really is Net::Telnet, then you'll have to dive down:</p>
  
  <pre><code>$mech-&gt;repl-&gt;repl-&gt;client-&gt;{telnet}-&gt;timeout($new_timeout);
  </code></pre>
  
  <p>** Update** <strong>so the questions are</strong>: i mage usage of ** Net::Telnet:** which is in  the Perl-Core  </p>
  
  <p>@ Alexandr Ciornii: thx for the hint! subsequently i would do it like this
   <strong>use: Net::Telnet;</strong> but if it is not in the core then i cannot go like this. 
  @ Daxim: $ corelist Net::Telnet␤␤Net::Telnet was not in CORE - that means i cannot go like above</p>
  
  <p><strong>btw</strong>: like Øyvind Skaar mentioned: With that many url's we have to expect that some will fail and handle that. For example, we put the failed ones in an array or hash and retry them X times.</p>