title: How to fork and read from multiple child processes?
body: <p>my main goal is to do some (externally depending time expensive) work for a large list of objects. For that, if I do it straight forward, it takes a lot of time. So I decided to go to parallel mode and fork some (4-8, let's see) child processes each of which does the job for a smaller set of objects. In the main (parent) process I wanted to printout the same overall statistic information of the progress I had for the one-process version.</p>
  
  <p>However, when I fork 4 child processes and do some work in it, I can see that they are alive, but only one of them is actually doing something and sending back information to the parent.</p>
  
  <p>Here's the code I've done so far - the time consuming part is mocked with a random usleep, which simulates its behaviour quite well.</p>
  
  <pre><code>#!/usr/bin/env perl
  use strict;
  use warnings;
  
  use DateTime;
  use DateTime::Format::HTTP;
  use Time::HiRes;
  
  my @to_be_processed = (1..300000);
  my @queues;
  my $nprocs = 4;
  
  my $parent_from_child;
  my @child_from_parent;
  my @child_to_parent;
  
  $SIG{CHLD} = 'IGNORE';
  $|=1; # autoflush
  
  my %stat = (
      total           =&gt; scalar(@to_be_processed),
      processed       =&gt; 0,
      time_started    =&gt; [Time::HiRes::gettimeofday],
  );
  
  # divide the list into queues for each subprocess
  for (my $i = 0; $i &lt; $stat{total}; $i++ ) {
      my $queue = $i % $nprocs;
      push @{$queues[$queue]}, $to_be_processed[$i];
  }
  
  # for progress simulation
  srand (time ^ $$);
  
  for (my $proc = 0; $proc &lt; $nprocs; $proc++) {
  
      # set up the pipes
      pipe $parent_from_child, $child_to_parent[$proc]        or die "pipe failed - $!";
  
      # fork
      defined(my $pid = fork) or die "fork failed - $!";
  
      if ($pid) {
          # parent
          close $child_to_parent[$proc];
          printf("[%u] parent says: child %u created with pid %u\n", $$, $proc, $pid);
      }
      else {
          # child
          close $parent_from_child;
          open(STDOUT, "&gt;&amp;=" . fileno($child_to_parent[$proc]))   or die "open failed - $!";
  
          warn(sprintf("[%u] child alive with %u entries\n", $$, scalar(@{$queues[$proc]})));
  
          foreach my $id (@{$queues[$proc]}) {
              printf("START: %s\n", $id);
  
              # simulation of progress
              my $random_microseconds = int(rand(3000000))+200000;
              warn(sprintf("[%u] child 'works' for %u microseconds", $$, $random_microseconds));
              Time::HiRes::usleep( $random_microseconds );
  
              printf("DONE\n")
          }
          exit(0);
      }
  }
  
  # parent: receive data from children and print overall statistics
  while (&lt;$parent_from_child&gt;) {
      chomp(my $line = $_);
  
      if ($line =~ m/^START: (\S+)/) {
          my ($id) = @_;
  
          printf("%6u/%6u", $stat{processed}, $stat{total});
          if ($stat{time_avg}) {
              my $remaining = ($stat{total} - $stat{processed}) * $stat{time_avg};
              my $eta = DateTime-&gt;from_epoch( epoch =&gt; time + $remaining );
              $eta-&gt;set_time_zone('Europe/Berlin');
              printf(" (ETA %s)", DateTime::Format::HTTP-&gt;format_isoz($eta));
          }
          printf("\r");
      }
      elsif ($line =~ /^DONE/) {
          $stat{processed}++;
          $stat{time_processed} = Time::HiRes::tv_interval( $stat{time_started} );
          $stat{time_avg}       = $stat{time_processed} / $stat{processed};
      }
      else {
          printf("%s\n", $line);
      }
  }
  </code></pre>
  
  <p>Normally the warn should be eliminated.
  If you run it you should see that only one child works.
  My question is: why? Where is my mistake and how can I get all of them doing the job?</p>
  
  <p>Thanks
  K.</p>