问:
Parent doesn't wait for child processes to finish despite reaping
I am fully aware that there are tonnes of articles explaining the inner workings of parent-child process dynamics. I have gone through them and got my stuff working as  I want it to function, almost. But there is one thing which is bugging me out and I am not able to understand it despite multiple tries.

Problem: Despite reaping the children, main is not waiting for all children to finish and exits prematurely. I believe I did make a proper exit from the child process and I have installed the REAPER in the child process - so how is main exiting before the child finishes?

Not looking for a solution here - but I need a new direction where I could bang my head for the next week. As of now - I feel I have exhausted my options and tried a lot many things but to no avail.

Some background about what I am trying to achieve:

All in all - I want all the children to finish, and only then I want to proceed to do something further. Each child process spawns a bunch of threads and those threads are properly joined by the said child process which then proceeds to make an exit with exit(0).

The additional hoopla you might observe in the program is nothing but our requirement where we are to hit 5 APIs (engines) but only with a fixed batch size, say 10 for each, at a time. I launch child process for each engine and launch thread for each request - and then I wait for all threads to finish, join them, and only then child process exits. Only now I could deposit the next batch of requests to the same engine, and I do this for all engines till I exhaust my total requests, say 10000.

Each request may take anywhere between 1 second to 2 hours - basically they are CSV reports being fetched from an HTTP API.

My issue is that when I have exhausted my total set of requests - I am not able to wait make the MAIN wait till all child processes have finished. This is weird and is the issue which I am trying to tackle.

Any ideas?

My Program OUTPUT:

```<code>[compuser@lenovoe470:little-stuff]$  perl 07--20190526-batch-processing-using-threads-with-busy-pool-detection-2.pl 12
26710: STARTING TASKS IN BATCHES
26710: RUNNING batch_engine 1_e1 tasks (1 2)
26710: RUNNING batch_engine 2_e2 tasks (3 4)
26710: RUNNING batch_engine 3_e3 tasks (5 6 7)
26710: BUSY_ENGINE: e1.
26710: BUSY_ENGINE: e2.
26710: BUSY_ENGINE: e3.
26710: BUSY_ENGINE: e1.
26710: BUSY_ENGINE: e2.
26710:26712: TASK_ORCHESTRATOR: >> finished batch_engine (2_e2) tasks (3 4)
26710: PID (26712) has finished with status (0). updating proc hash
26710: BUSY_ENGINE: e3.
26710:26713: TASK_ORCHESTRATOR: >> finished batch_engine (3_e3) tasks (5 6 7)
26710:26711: TASK_ORCHESTRATOR: >> finished batch_engine (1_e1) tasks (1 2)
26710: PID (26713) has finished with status (0). updating proc hash
26710: BUSY_ENGINE: e1.
26710: PID (26711) has finished with status (0). updating proc hash
26710: RUNNING batch_engine 4_e2 tasks (8 9)
26710: RUNNING batch_engine 5_e3 tasks (10 11 12)
26710: FINISHED TASKS IN BATCHES
[compuser@lenovoe470:little-stuff]$  1:26722: TASK_ORCHESTRATOR: >> finished batch_engine (5_e3) tasks (10 11 12)
1:26721: TASK_ORCHESTRATOR: >> finished batch_engine (4_e2) tasks (8 9)
</code>```

In above output:

Running batch_engine means I am running a batch of numbered tasks.

BUSY_ENGINE means that the endpoint/engine is busy as it is already busy handling max batch size of requests. I need to wait.

finished batch_engine means that the child process has finished processing the given batch of requests for a particular engine/end-point. It exits and main detects that the current engine is now free and next batch can be en-queued

if we see the last 2 lines, its evident that the child processes' output has spilled over & main exited prematurely without waiting for the running children. Why? any help?

My program:

```<code>#!/usr/bin/env perl

use strict;
use warnings;
use Data::Dumper;
use POSIX ':sys_wait_h';
use Thread qw(async);

STDOUT->autoflush(1);

# doesn't work
  sub reaper {
    my $reaped;
    while (($reaped = waitpid (-1,&WNOHANG) > 0)) {
      print "$$: reaped: $reaped\n";
      sleep(1);
    }
    $SIG{CHLD} = \&reaper;
  }
# doesn't work

my @total_tasks = (1 .. shift || 9);
my @engines = (qw/e1 e2 e3/);
my $sizes = { e1 => 2, e2 => 2, e3 => 3, };

my $proc_hash;
my $global_string = "ENGINE";

# source: https://duyanghao.github.io/ways_avoid_zombie_process/
#
  sub REAPER {
    local ($!, $?);
    while ( (my $reaped_pid = waitpid(-1, WNOHANG)) > 0 ) {
      if ( WIFEXITED($?) )
      {
        # my
        my $ret_code = WEXITSTATUS($?);
        print "$$: PID ($reaped_pid) has finished with status ($ret_code). updating proc hash\n";
        my $engine_name = $proc_hash->{$reaped_pid};
        delete ($proc_hash->{$reaped_pid});
        delete ($proc_hash->{$engine_name});
        # my

        # original
        #my $ret_code = WEXITSTATUS($?);
        #print "child process:$pid exit with code:$ret_code\n";
        # original
      }
    }
  }
#

$SIG{CHLD} = \&REAPER;

sub random_sleep_time {
  return (int(rand(5)+1))
  #return (sprintf "%.2f",(rand(1)+1))
}

sub task_runner {
  my @args = @_;
  my ($batch_engine, $task) = ($args[0]->[0],$args[0]->[1]);
  STDOUT->autoflush(1);
  my $task_time = random_sleep_time();
  sleep ($task_time);
  threads->exit(0);
  #print "$$:".(threads->tid()).": TASK_RUNNER: $global_string ($batch_engine) task ($task) finished in $task_time seconds\n";
  #return;
};

sub task_orchestrator {
  my ($batch_engine, @tasks) = @_;
  my $engine = (split (/_/,$batch_engine))[1];
  my $task_orch_pid = fork();
  die "Failed to fork task_orchestrator\n" if not defined $task_orch_pid;

  if ($task_orch_pid != 0) {
    $proc_hash->{$engine} = $task_orch_pid;
    $proc_hash->{$task_orch_pid} = $engine;
  }

  if ($task_orch_pid == 0) {
    STDOUT->autoflush(1);
    my @tids;
    for (my $i=1 ; $i <= $#tasks ; $i++) { push (@tids,$i) }
    foreach my $task_number (0 .. $#tasks) {
      $tids [$task_number] = threads->create (
        \&task_runner,[$batch_engine,$tasks [$task_number]]
      );
    }
    my $ppid = getppid();
    foreach my $tid (@tids) {$tid->join()}
    print "$ppid:$$: TASK_ORCHESTRATOR: >> finished batch_engine ($batch_engine) tasks (@tasks)\n";
    exit (0);
  }
}

sub update_proc_hash {
  my $finished_pid = waitpid (-1, POSIX->WNOHANG);
  if ($finished_pid > 0) {
    print "$$: PID ($finished_pid) has finished. updating proc hash\n";
    my $engine_name = $proc_hash->{$finished_pid};
    delete ($proc_hash->{$finished_pid});
    delete ($proc_hash->{$engine_name});
  }
}

my $batch=1;
print "$$: STARTING TASKS IN BATCHES\n";
while (@total_tasks) {
  foreach my $engine (@engines) {
    update_proc_hash();
    if (exists $proc_hash->{$engine}) {
      print "$$: BUSY_ENGINE: $engine.\n";
      sleep (1);
      next;
    }
    else {
      my @engine_tasks;
      my $engine_max_tasks = $sizes->{$engine};
      while ($engine_max_tasks-- != 0) {
        my $task = shift @total_tasks;
        push (@engine_tasks,$task) if $task;
      }
      if (@engine_tasks) {
        my $batch_engine = $batch.'_'.$engine;
        print "$$: RUNNING batch_engine $batch_engine tasks (@engine_tasks)\n";
        task_orchestrator ("$batch_engine",@engine_tasks);
        $batch++;
      }
    }
  }
}

REAPER();

print "$$: FINISHED TASKS IN BATCHES\n";

__END__
</code>```

Update after 3 days: Thank you SO community. Once again, I am grateful to all of you who have taken out their time to look into this and helped spot and correct the problem. Thank you so much.

Allow me to share the new output with the final program for everyone's reference.

OUTPUT after using the fix:

```<code>User@Host:/cygdrive/c/bash-home> perl test.pl
22044: STARTING TASKS IN BATCHES
22044: MAIN: engine (e1) is RUNNING batch #1 tasks: (1 2)
22044: MAIN: engine (e2) is RUNNING batch #2 tasks: (3 4 5)
22044: MAIN: engine (e3) is RUNNING batch #3 tasks: (6 7)
41456: TASK_RUNNER: engine (e1) finished batch #1 task #1 in (1.80) seconds
41456: TASK_RUNNER: engine (e1) finished batch #1 task #2 in (1.31) seconds
41456: TASK_ORCHESTRATOR: engine (e1) finished batch #1 tasks in (1.00) seconds.
22044: REAPER: TASK_ORCHESTRATOR pid (41456) has finished with status (0).
18252: TASK_RUNNER: engine (e2) finished batch #2 task #3 in (1.04) seconds
18252: TASK_RUNNER: engine (e2) finished batch #2 task #4 in (1.91) seconds
18252: TASK_RUNNER: engine (e2) finished batch #2 task #5 in (1.63) seconds
18252: TASK_ORCHESTRATOR: engine (e2) finished batch #2 tasks in (1.00) seconds.
22044: REAPER: TASK_ORCHESTRATOR pid (18252) has finished with status (0).
14544: TASK_RUNNER: engine (e3) finished batch #3 task #6 in (1.42) seconds
14544: TASK_RUNNER: engine (e3) finished batch #3 task #7 in (1.84) seconds
14544: TASK_ORCHESTRATOR: engine (e3) finished batch #3 tasks in (1.00) seconds.
22044: REAPER: TASK_ORCHESTRATOR pid (14544) has finished with status (0).
22044: MAIN: engine (e1) is RUNNING batch #4 tasks: (8 9)
22044: MAIN: engine (e2) is RUNNING batch #5 tasks: (10)
37612: TASK_RUNNER: engine (e1) finished batch #4 task #8 in (1.19) seconds
37612: TASK_RUNNER: engine (e1) finished batch #4 task #9 in (1.31) seconds
37612: TASK_ORCHESTRATOR: engine (e1) finished batch #4 tasks in (1.00) seconds.
16300: TASK_RUNNER: engine (e2) finished batch #5 task #10 in (1.53) seconds
16300: TASK_ORCHESTRATOR: engine (e2) finished batch #5 tasks in (1.00) seconds.
22044: ALL ORCHESTRATORS HAVE FINISHED
22044: FINISHED TASKS IN BATCHES
</code>```

FINAL Working PROGRAM:

```<code>#!/usr/bin/env perl

use strict;
use warnings;
use Data::Dumper;
use POSIX ':sys_wait_h';
use threads;

STDOUT->autoflush(1);

my @total_tasks = (1 .. 10);
my $sleep_time = 1;
my @engines = (qw/e1 e2 e3/);
my $sizes = {
  e1 => 2,
  e2 => 3,
  e3 => 2,
};

my $proc_hash;
my $global_string = "engine";

sub REAPER {
  local ($!, $?);
  while ( (my $reaped_pid = waitpid(-1, WNOHANG)) > 0 ) {
    if ( WIFEXITED($?) ) {
      my $ret_code = WEXITSTATUS($?);
      print "$$: REAPER: TASK_ORCHESTRATOR pid ($reaped_pid) has finished with status ($ret_code).\n";
      my $engine_name = $proc_hash->{$reaped_pid};
      delete ($proc_hash->{$reaped_pid});
      delete ($proc_hash->{$engine_name});
    }
  }
}

$SIG{CHLD} = \&REAPER;

sub random_sleep_time { return sprintf ("%.2f",(rand ($sleep_time||5) + 1)) }

sub task_runner {
  STDOUT->autoflush(1);
  my @args = @_;
  my ($batch_engine, $task) = ($args[0]->[0],$args[0]->[1]);
  my ($batch, $engine) = split (/_/,$batch_engine);
  my $task_time = random_sleep_time();
  sleep ($task_time);
  print "$$: TASK_RUNNER: $global_string ($engine) finished batch #$batch task #$task in ($task_time) seconds\n";
  threads->exit(0);
};

sub task_orchestrator {
  my ($batch_engine, @tasks) = @_;
  my ($batch, $engine) = split (/_/,$batch_engine);
  my $task_orch_pid = fork();
  die "Failed to fork task_orchestrator\n" if not defined $task_orch_pid;

  if ($task_orch_pid != 0) {
    $proc_hash->{$engine} = $task_orch_pid;
    $proc_hash->{$task_orch_pid} = $engine;
  }

  if ($task_orch_pid == 0) {
    STDOUT->autoflush(1);
    my @tids;
    my $start_time = time;
    for (my $i=1 ; $i <= $#tasks ; $i++) { push (@tids,$i) }
    foreach my $task_number (0 .. $#tasks) {
      $tids [$task_number] = threads->create (
        \&task_runner,[$batch_engine,$tasks [$task_number]]
      );
    }
    foreach my $tid (@tids) {$tid->join()}
    my $end_time = time;
    my $total_time = sprintf ("%.2f",($end_time - $start_time));
    print "$$: TASK_ORCHESTRATOR: engine ($engine) finished batch #$batch tasks in ($total_time) seconds.\n";
    exit (0);
  }
}

my $batch=1;
print "$$: STARTING TASKS IN BATCHES\n";
while (@total_tasks)
{
  foreach my $engine (@engines)
  {
    if (exists $proc_hash->{$engine})
    {
      sleep (1);
      next;
    }
    else
    {
      my @engine_tasks;
      my $engine_max_tasks = $sizes->{$engine};
      while ($engine_max_tasks-- != 0)
      {
        my $task = shift @total_tasks;
        push (@engine_tasks,$task) if $task;
      }
      if (@engine_tasks)
      {
        my $batch_engine = $batch.'_'.$engine;
        print "$$: MAIN: engine ($engine) is RUNNING batch #$batch tasks: (@engine_tasks)\n";
        task_orchestrator ($batch_engine,@engine_tasks);
        $batch++;
      }
    }
  }
}

# All 3 below work properly
#sleep (.2) while ((waitpid(-1, WNOHANG)) >= 0);
#sleep (.2) while ((waitpid(-1, WNOHANG)) != -1);
sleep (.2) while ((waitpid(-1, WNOHANG)) > -1);

print "$$: ALL ORCHESTRATORS HAVE FINISHED\n";
print "$$: FINISHED TASKS IN BATCHES\n";
__END__
</code>```

答:
The waitpid

can return 0 if there are child processes matching PID but none have terminated yet

With -1 this applies to any child process and so your code with multiple children will surely encounter a zero return from the non-blocking waitpid in REAPER; this is precisely how we get to wait as long as there are non-terminated child processes.  But your while loop exits at first such zero.

One way about this is to poll for non-negative returns

```<code>use warnings;
use strict;
use feature 'say';

use POSIX ':sys_wait_h';
use Time::HiRes qw(sleep) ;

for (1..4) {
    my $pid = fork // die "Can't fork: $!";
    if ($pid == 0) {
        sleep rand 4;
        say "\tkid $$ exiting";
        exit;
    };
};

while ( (my $kid = waitpid -1, WNOHANG) > -1 ) {
    say "got $kid" if $kid > 0;
    sleep 0.2;
}
</code>```

Prints

```
        kid 12687 exiting
got 12687
        kid 12690 exiting
got 12690
        kid 12689 exiting
got 12689
        kid 12688 exiting
got 12688
```

Please adjust the polling period as suitable.  Note that since this catches any child processes it is possible for it to interfere with yet other forks, if there were any unwaited ones by that point.

Or you can block with the wait

```<code>while ( (my $kid = waitpid -1, 0) > -1 ) {
    say "got $kid";
}
</code>```

where you can now also do > 0, as there'll be no 0 returns here since the call blocks. While we only need the loop to terminate once -1 comes back (no more processes out there), as before.

The major difference is that the block executes only once a child process actually exited, so if you need to keep tabs on what some long-running children are doing (and perhaps limit their run times or protect against hung jobs) that is not as easy in this form; you want a non-blocking operation for that.

Note that some of details, in particular relating to returns, may vary across systems.

The naive version of this is to wait only for these specific PIDs, collected as you fork

```<code>foreach my $pid (@pids) {
    my $gone = waitpid $pid, 0;
    say "Process $gone exited with $?" if $gone > 0;  # -1 if reaped already
}
</code>```

which blocks with waitpid for each process. The problem with this is that if one process runs much longer than others (or hangs) this loop will be stuck waiting at it. And, just in general, we'd rather have children processes reaped as they exit than in the order in which they were started.

Body:
I am fully aware that there are tonnes of articles explaining the inner workings of parent-child process dynamics. I have gone through them and got my stuff working as  I want it to function, almost. But there is one thing which is bugging me out and I am not able to understand it despite multiple tries.

Problem: Despite reaping the children, main is not waiting for all children to finish and exits prematurely. I believe I did make a proper exit from the child process and I have installed the REAPER in the child process - so how is main exiting before the child finishes?

Not looking for a solution here - but I need a new direction where I could bang my head for the next week. As of now - I feel I have exhausted my options and tried a lot many things but to no avail.

Some background about what I am trying to achieve:

All in all - I want all the children to finish, and only then I want to proceed to do something further. Each child process spawns a bunch of threads and those threads are properly joined by the said child process which then proceeds to make an exit with exit(0).

The additional hoopla you might observe in the program is nothing but our requirement where we are to hit 5 APIs (engines) but only with a fixed batch size, say 10 for each, at a time. I launch child process for each engine and launch thread for each request - and then I wait for all threads to finish, join them, and only then child process exits. Only now I could deposit the next batch of requests to the same engine, and I do this for all engines till I exhaust my total requests, say 10000.

Each request may take anywhere between 1 second to 2 hours - basically they are CSV reports being fetched from an HTTP API.

My issue is that when I have exhausted my total set of requests - I am not able to wait make the MAIN wait till all child processes have finished. This is weird and is the issue which I am trying to tackle.

Any ideas?

My Program OUTPUT:

[compuser@lenovoe470:little-stuff]$  perl 07--20190526-batch-processing-using-threads-with-busy-pool-detection-2.pl 12
26710: STARTING TASKS IN BATCHES
26710: RUNNING batch_engine 1_e1 tasks (1 2)
26710: RUNNING batch_engine 2_e2 tasks (3 4)
26710: RUNNING batch_engine 3_e3 tasks (5 6 7)
26710: BUSY_ENGINE: e1.
26710: BUSY_ENGINE: e2.
26710: BUSY_ENGINE: e3.
26710: BUSY_ENGINE: e1.
26710: BUSY_ENGINE: e2.
26710:26712: TASK_ORCHESTRATOR: >> finished batch_engine (2_e2) tasks (3 4)
26710: PID (26712) has finished with status (0). updating proc hash
26710: BUSY_ENGINE: e3.
26710:26713: TASK_ORCHESTRATOR: >> finished batch_engine (3_e3) tasks (5 6 7)
26710:26711: TASK_ORCHESTRATOR: >> finished batch_engine (1_e1) tasks (1 2)
26710: PID (26713) has finished with status (0). updating proc hash
26710: BUSY_ENGINE: e1.
26710: PID (26711) has finished with status (0). updating proc hash
26710: RUNNING batch_engine 4_e2 tasks (8 9)
26710: RUNNING batch_engine 5_e3 tasks (10 11 12)
26710: FINISHED TASKS IN BATCHES
[compuser@lenovoe470:little-stuff]$  1:26722: TASK_ORCHESTRATOR: >> finished batch_engine (5_e3) tasks (10 11 12)
1:26721: TASK_ORCHESTRATOR: >> finished batch_engine (4_e2) tasks (8 9)

In above output:

Running batch_engine means I am running a batch of numbered tasks.

BUSY_ENGINE means that the endpoint/engine is busy as it is already busy handling max batch size of requests. I need to wait.

finished batch_engine means that the child process has finished processing the given batch of requests for a particular engine/end-point. It exits and main detects that the current engine is now free and next batch can be en-queued

if we see the last 2 lines, its evident that the child processes' output has spilled over & main exited prematurely without waiting for the running children. Why? any help?

My program:

#!/usr/bin/env perl

use strict;
use warnings;
use Data::Dumper;
use POSIX ':sys_wait_h';
use Thread qw(async);

STDOUT->autoflush(1);

# doesn't work
  sub reaper {
    my $reaped;
    while (($reaped = waitpid (-1,&WNOHANG) > 0)) {
      print "$$: reaped: $reaped\n";
      sleep(1);
    }
    $SIG{CHLD} = \&reaper;
  }
# doesn't work

my @total_tasks = (1 .. shift || 9);
my @engines = (qw/e1 e2 e3/);
my $sizes = { e1 => 2, e2 => 2, e3 => 3, };

my $proc_hash;
my $global_string = "ENGINE";

# source: https://duyanghao.github.io/ways_avoid_zombie_process/
#
  sub REAPER {
    local ($!, $?);
    while ( (my $reaped_pid = waitpid(-1, WNOHANG)) > 0 ) {
      if ( WIFEXITED($?) )
      {
        # my
        my $ret_code = WEXITSTATUS($?);
        print "$$: PID ($reaped_pid) has finished with status ($ret_code). updating proc hash\n";
        my $engine_name = $proc_hash->{$reaped_pid};
        delete ($proc_hash->{$reaped_pid});
        delete ($proc_hash->{$engine_name});
        # my

        # original
        #my $ret_code = WEXITSTATUS($?);
        #print "child process:$pid exit with code:$ret_code\n";
        # original
      }
    }
  }
#

$SIG{CHLD} = \&REAPER;

sub random_sleep_time {
  return (int(rand(5)+1))
  #return (sprintf "%.2f",(rand(1)+1))
}

sub task_runner {
  my @args = @_;
  my ($batch_engine, $task) = ($args[0]->[0],$args[0]->[1]);
  STDOUT->autoflush(1);
  my $task_time = random_sleep_time();
  sleep ($task_time);
  threads->exit(0);
  #print "$$:".(threads->tid()).": TASK_RUNNER: $global_string ($batch_engine) task ($task) finished in $task_time seconds\n";
  #return;
};

sub task_orchestrator {
  my ($batch_engine, @tasks) = @_;
  my $engine = (split (/_/,$batch_engine))[1];
  my $task_orch_pid = fork();
  die "Failed to fork task_orchestrator\n" if not defined $task_orch_pid;

  if ($task_orch_pid != 0) {
    $proc_hash->{$engine} = $task_orch_pid;
    $proc_hash->{$task_orch_pid} = $engine;
  }

  if ($task_orch_pid == 0) {
    STDOUT->autoflush(1);
    my @tids;
    for (my $i=1 ; $i <= $#tasks ; $i++) { push (@tids,$i) }
    foreach my $task_number (0 .. $#tasks) {
      $tids [$task_number] = threads->create (
        \&task_runner,[$batch_engine,$tasks [$task_number]]
      );
    }
    my $ppid = getppid();
    foreach my $tid (@tids) {$tid->join()}
    print "$ppid:$$: TASK_ORCHESTRATOR: >> finished batch_engine ($batch_engine) tasks (@tasks)\n";
    exit (0);
  }
}

sub update_proc_hash {
  my $finished_pid = waitpid (-1, POSIX->WNOHANG);
  if ($finished_pid > 0) {
    print "$$: PID ($finished_pid) has finished. updating proc hash\n";
    my $engine_name = $proc_hash->{$finished_pid};
    delete ($proc_hash->{$finished_pid});
    delete ($proc_hash->{$engine_name});
  }
}

my $batch=1;
print "$$: STARTING TASKS IN BATCHES\n";
while (@total_tasks) {
  foreach my $engine (@engines) {
    update_proc_hash();
    if (exists $proc_hash->{$engine}) {
      print "$$: BUSY_ENGINE: $engine.\n";
      sleep (1);
      next;
    }
    else {
      my @engine_tasks;
      my $engine_max_tasks = $sizes->{$engine};
      while ($engine_max_tasks-- != 0) {
        my $task = shift @total_tasks;
        push (@engine_tasks,$task) if $task;
      }
      if (@engine_tasks) {
        my $batch_engine = $batch.'_'.$engine;
        print "$$: RUNNING batch_engine $batch_engine tasks (@engine_tasks)\n";
        task_orchestrator ("$batch_engine",@engine_tasks);
        $batch++;
      }
    }
  }
}

REAPER();

print "$$: FINISHED TASKS IN BATCHES\n";

__END__

Update after 3 days: Thank you SO community. Once again, I am grateful to all of you who have taken out their time to look into this and helped spot and correct the problem. Thank you so much.

Allow me to share the new output with the final program for everyone's reference.

OUTPUT after using the fix:

User@Host:/cygdrive/c/bash-home> perl test.pl
22044: STARTING TASKS IN BATCHES
22044: MAIN: engine (e1) is RUNNING batch #1 tasks: (1 2)
22044: MAIN: engine (e2) is RUNNING batch #2 tasks: (3 4 5)
22044: MAIN: engine (e3) is RUNNING batch #3 tasks: (6 7)
41456: TASK_RUNNER: engine (e1) finished batch #1 task #1 in (1.80) seconds
41456: TASK_RUNNER: engine (e1) finished batch #1 task #2 in (1.31) seconds
41456: TASK_ORCHESTRATOR: engine (e1) finished batch #1 tasks in (1.00) seconds.
22044: REAPER: TASK_ORCHESTRATOR pid (41456) has finished with status (0).
18252: TASK_RUNNER: engine (e2) finished batch #2 task #3 in (1.04) seconds
18252: TASK_RUNNER: engine (e2) finished batch #2 task #4 in (1.91) seconds
18252: TASK_RUNNER: engine (e2) finished batch #2 task #5 in (1.63) seconds
18252: TASK_ORCHESTRATOR: engine (e2) finished batch #2 tasks in (1.00) seconds.
22044: REAPER: TASK_ORCHESTRATOR pid (18252) has finished with status (0).
14544: TASK_RUNNER: engine (e3) finished batch #3 task #6 in (1.42) seconds
14544: TASK_RUNNER: engine (e3) finished batch #3 task #7 in (1.84) seconds
14544: TASK_ORCHESTRATOR: engine (e3) finished batch #3 tasks in (1.00) seconds.
22044: REAPER: TASK_ORCHESTRATOR pid (14544) has finished with status (0).
22044: MAIN: engine (e1) is RUNNING batch #4 tasks: (8 9)
22044: MAIN: engine (e2) is RUNNING batch #5 tasks: (10)
37612: TASK_RUNNER: engine (e1) finished batch #4 task #8 in (1.19) seconds
37612: TASK_RUNNER: engine (e1) finished batch #4 task #9 in (1.31) seconds
37612: TASK_ORCHESTRATOR: engine (e1) finished batch #4 tasks in (1.00) seconds.
16300: TASK_RUNNER: engine (e2) finished batch #5 task #10 in (1.53) seconds
16300: TASK_ORCHESTRATOR: engine (e2) finished batch #5 tasks in (1.00) seconds.
22044: ALL ORCHESTRATORS HAVE FINISHED
22044: FINISHED TASKS IN BATCHES

FINAL Working PROGRAM:

#!/usr/bin/env perl

use strict;
use warnings;
use Data::Dumper;
use POSIX ':sys_wait_h';
use threads;

STDOUT->autoflush(1);

my @total_tasks = (1 .. 10);
my $sleep_time = 1;
my @engines = (qw/e1 e2 e3/);
my $sizes = {
  e1 => 2,
  e2 => 3,
  e3 => 2,
};

my $proc_hash;
my $global_string = "engine";

sub REAPER {
  local ($!, $?);
  while ( (my $reaped_pid = waitpid(-1, WNOHANG)) > 0 ) {
    if ( WIFEXITED($?) ) {
      my $ret_code = WEXITSTATUS($?);
      print "$$: REAPER: TASK_ORCHESTRATOR pid ($reaped_pid) has finished with status ($ret_code).\n";
      my $engine_name = $proc_hash->{$reaped_pid};
      delete ($proc_hash->{$reaped_pid});
      delete ($proc_hash->{$engine_name});
    }
  }
}

$SIG{CHLD} = \&REAPER;

sub random_sleep_time { return sprintf ("%.2f",(rand ($sleep_time||5) + 1)) }

sub task_runner {
  STDOUT->autoflush(1);
  my @args = @_;
  my ($batch_engine, $task) = ($args[0]->[0],$args[0]->[1]);
  my ($batch, $engine) = split (/_/,$batch_engine);
  my $task_time = random_sleep_time();
  sleep ($task_time);
  print "$$: TASK_RUNNER: $global_string ($engine) finished batch #$batch task #$task in ($task_time) seconds\n";
  threads->exit(0);
};

sub task_orchestrator {
  my ($batch_engine, @tasks) = @_;
  my ($batch, $engine) = split (/_/,$batch_engine);
  my $task_orch_pid = fork();
  die "Failed to fork task_orchestrator\n" if not defined $task_orch_pid;

  if ($task_orch_pid != 0) {
    $proc_hash->{$engine} = $task_orch_pid;
    $proc_hash->{$task_orch_pid} = $engine;
  }

  if ($task_orch_pid == 0) {
    STDOUT->autoflush(1);
    my @tids;
    my $start_time = time;
    for (my $i=1 ; $i <= $#tasks ; $i++) { push (@tids,$i) }
    foreach my $task_number (0 .. $#tasks) {
      $tids [$task_number] = threads->create (
        \&task_runner,[$batch_engine,$tasks [$task_number]]
      );
    }
    foreach my $tid (@tids) {$tid->join()}
    my $end_time = time;
    my $total_time = sprintf ("%.2f",($end_time - $start_time));
    print "$$: TASK_ORCHESTRATOR: engine ($engine) finished batch #$batch tasks in ($total_time) seconds.\n";
    exit (0);
  }
}

my $batch=1;
print "$$: STARTING TASKS IN BATCHES\n";
while (@total_tasks)
{
  foreach my $engine (@engines)
  {
    if (exists $proc_hash->{$engine})
    {
      sleep (1);
      next;
    }
    else
    {
      my @engine_tasks;
      my $engine_max_tasks = $sizes->{$engine};
      while ($engine_max_tasks-- != 0)
      {
        my $task = shift @total_tasks;
        push (@engine_tasks,$task) if $task;
      }
      if (@engine_tasks)
      {
        my $batch_engine = $batch.'_'.$engine;
        print "$$: MAIN: engine ($engine) is RUNNING batch #$batch tasks: (@engine_tasks)\n";
        task_orchestrator ($batch_engine,@engine_tasks);
        $batch++;
      }
    }
  }
}

# All 3 below work properly
#sleep (.2) while ((waitpid(-1, WNOHANG)) >= 0);
#sleep (.2) while ((waitpid(-1, WNOHANG)) != -1);
sleep (.2) while ((waitpid(-1, WNOHANG)) > -1);

print "$$: ALL ORCHESTRATORS HAVE FINISHED\n";
print "$$: FINISHED TASKS IN BATCHES\n";
__END__

Body:
The waitpid

can return 0 if there are child processes matching PID but none have terminated yet

With -1 this applies to any child process and so your code with multiple children will surely encounter a zero return from the non-blocking waitpid in REAPER; this is precisely how we get to wait as long as there are non-terminated child processes.  But your while loop exits at first such zero.

One way about this is to poll for non-negative returns

use warnings;
use strict;
use feature 'say';

use POSIX ':sys_wait_h';
use Time::HiRes qw(sleep) ;

for (1..4) {
    my $pid = fork // die "Can't fork: $!";
    if ($pid == 0) {
        sleep rand 4;
        say "\tkid $$ exiting";
        exit;
    };
};

while ( (my $kid = waitpid -1, WNOHANG) > -1 ) {
    say "got $kid" if $kid > 0;
    sleep 0.2;
}

Prints

        kid 12687 exiting
got 12687
        kid 12690 exiting
got 12690
        kid 12689 exiting
got 12689
        kid 12688 exiting
got 12688

Please adjust the polling period as suitable.  Note that since this catches any child processes it is possible for it to interfere with yet other forks, if there were any unwaited ones by that point.

Or you can block with the wait

while ( (my $kid = waitpid -1, 0) > -1 ) {
    say "got $kid";
}

where you can now also do > 0, as there'll be no 0 returns here since the call blocks. While we only need the loop to terminate once -1 comes back (no more processes out there), as before.

The major difference is that the block executes only once a child process actually exited, so if you need to keep tabs on what some long-running children are doing (and perhaps limit their run times or protect against hung jobs) that is not as easy in this form; you want a non-blocking operation for that.

Note that some of details, in particular relating to returns, may vary across systems.

The naive version of this is to wait only for these specific PIDs, collected as you fork

foreach my $pid (@pids) {
    my $gone = waitpid $pid, 0;
    say "Process $gone exited with $?" if $gone > 0;  # -1 if reaped already
}

which blocks with waitpid for each process. The problem with this is that if one process runs much longer than others (or hangs) this loop will be stuck waiting at it. And, just in general, we'd rather have children processes reaped as they exit than in the order in which they were started.


来源: stackexchange

问:
update multiple rows after insert in codeigniter
After insert I want to update field with ($id.'mytext') where in db that field is empty for all rows.

```<code>table name: peca
columns:
id -autoincrement
A -varchar user insert
CB -varchar auto insert with update
</code>```

```<code>MODEL WILL RETURN ALL ROWS WHERE CB=empty

function model1() {
        $this->db->select('*');
        $this->db->from('peca');
        $this->db->where('CB', '');
        //$this->db->where('id', $fileid);
        $query = $this->db->get();
        if ($query) {
            return $query->result();
        } else {
            return false;
        }
    }
</code>```

```<code>MODEL WILL update in db where CB=empty
function model2($dados = NULL) {
        if ($dados !== NULL) {
        //  extract($dados);
         $this->db->where('CB', '');
            $this->db->update('peca', $dados);
            return true;
        } else {
            return false;
        }
    }
</code>```

```<code>CONTROLLER

 $this->load->model('model_peca');
            $resultadocadastropeca = $this->model_peca->model1($id);
            $data = 'id' => $id;
            $appointment = array('codigoean' => $data.'.PECA.');
            $appointment = $this->model_peca->model2($appointment);

</code>```

START TABLE
Previous values inserted from import so CB can only be generated after id exists

```<code>id|CB      |
22|        |
31|        |
</code>```

RESULTS
I'm changing CB to .PECA. in all rows where CB=empty but $id for each row is not passing

```<code>id|CB    |
22|.PECA.|
31|.PECA.|
</code>```

EXPECTED

```<code>id|CB      |
22|22.PECA.|
31|31.PECA.|
</code>```

答:
I still don't fully understand what you want to do, but if you want to fetch something where a column is empty or null:

$this->db->where("WHERE  yourColumn IS NULL OR yourColumn = ''");

As per your revised question, this should work:

```<code>$this->db->select('id');
$this->db->where('CB', '');
$q = $this->db->get('peca');
if ($q->num_rows() == 0) {
   die('nothing to do here');
}
$res = $q->result();
foreach ($res as $item) {
     $data[] = array('id' => $item->id, 'CB' => $item->id . '.PECA.');
}
$this->db->update_batch('peca', $data, 'id');
</code>```

ContentLicense: CC BY-SA 4.0

Body:
After insert I want to update field with ($id.'mytext') where in db that field is empty for all rows.

table name: peca
columns:
id -autoincrement
A -varchar user insert
CB -varchar auto insert with update

MODEL WILL RETURN ALL ROWS WHERE CB=empty

function model1() {
        $this->db->select('*');
        $this->db->from('peca');
        $this->db->where('CB', '');
        //$this->db->where('id', $fileid);
        $query = $this->db->get();
        if ($query) {
            return $query->result();
        } else {
            return false;
        }
    }

MODEL WILL update in db where CB=empty
function model2($dados = NULL) {
        if ($dados !== NULL) {
        //  extract($dados);
         $this->db->where('CB', '');
            $this->db->update('peca', $dados);
            return true;
        } else {
            return false;
        }
    }

CONTROLLER

 $this->load->model('model_peca');
            $resultadocadastropeca = $this->model_peca->model1($id);
            $data = 'id' => $id;
            $appointment = array('codigoean' => $data.'.PECA.');
            $appointment = $this->model_peca->model2($appointment);

START TABLE
Previous values inserted from import so CB can only be generated after id exists

id|CB      |
22|        |
31|        |

RESULTS
I'm changing CB to .PECA. in all rows where CB=empty but $id for each row is not passing

id|CB    |
22|.PECA.|
31|.PECA.|

EXPECTED

id|CB      |
22|22.PECA.|
31|31.PECA.|

Body:
I still don't fully understand what you want to do, but if you want to fetch something where a column is empty or null:

$this->db->where("WHERE  yourColumn IS NULL OR yourColumn = ''");

As per your revised question, this should work:

$this->db->select('id');
$this->db->where('CB', '');
$q = $this->db->get('peca');
if ($q->num_rows() == 0) {
   die('nothing to do here');
}
$res = $q->result();
foreach ($res as $item) {
     $data[] = array('id' => $item->id, 'CB' => $item->id . '.PECA.');
}
$this->db->update_batch('peca', $data, 'id');


问:
Rotating a 1d RGBA array
I'm working with a 1D pixel RGBA array that looks like this:

```<code>pixelArray =[0,0,0,255,0,0,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255];
</code>```

What this pixelArray corresponds to when drawn is 2 black pixels and 4 white pixels:

```<code>BB
WW
WW
</code>```

My goal is to rotate the order of the pixels inside the array so the picture when drawn will look like

```<code>BWW  or WWB
BWW     WWB
</code>```

Which means I need to convert the pixelArray to

```<code>rotatedPixelArray = [0,0,0,255,255,255,255,255,255,255,255,255,0,0,0,255,255,255,255,255,255,255,255,255]
</code>```

The above example is just that, an example. The actual rgba could represent any image and could have a length of 1 million +.
I've tried various algorithms like this, and converting to a 2d array and then rotating and flattening (which does work) but i'd like to avoid this because speed/memory is an issue.

答:
Maybe this could be helpful in p5.js for 90 degrees rotation (left and right) (the previous suggested solution seems to have a little 'asymmetric' error):</p>

```<code>function rotateRight(img){

  var w = img.width;
  var h = img.height;
  var index, indexr;
  var img2 = createImage(w, h);

  img.loadPixels();
  img2.loadPixels();

  indexr = 0;

  for (let x = 0; x < w; x++) {

    for(let y = h - 1; y >= 0; y--) {

      index = (x+y*w)*4;
      img2.pixels[indexr] = img.pixels[index];
      img2.pixels[indexr + 1] = img.pixels[index+1];
      img2.pixels[indexr + 2] = img.pixels[index+2];
      img2.pixels[indexr + 3] = img.pixels[index+3];

      indexr += 4;

    }
  }

  img.updatePixels();
  img2.updatePixels();

  return img2;

}

function rotateLeft(img){

  var w = img.width;
  var h = img.height;
  var index, indexr;
  var img2 = createImage(w, h);

  img.loadPixels();
  img2.loadPixels();

  indexr = 0;

  for (let x = w - 1; x >= 0; x--) {

    for(let y = 0; y < h; y++) {

      index = (x+y*w)*4;
      img2.pixels[indexr] = img.pixels[index];
      img2.pixels[indexr + 1] = img.pixels[index+1];
      img2.pixels[indexr + 2] = img.pixels[index+2];
      img2.pixels[indexr + 3] = img.pixels[index+3];

      indexr += 4;

    }
  }

  img.updatePixels();
  img2.updatePixels();

  return img2;

}
</code>```

Title: Rotating a 1d RGBA array

Body:
I'm working with a 1D pixel RGBA array that looks like this:

pixelArray =[0,0,0,255,0,0,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255];

What this pixelArray corresponds to when drawn is 2 black pixels and 4 white pixels:

BB
WW
WW

My goal is to rotate the order of the pixels inside the array so the picture when drawn will look like

BWW  or WWB
BWW     WWB

Which means I need to convert the pixelArray to

rotatedPixelArray = [0,0,0,255,255,255,255,255,255,255,255,255,0,0,0,255,255,255,255,255,255,255,255,255]

The above example is just that, an example. The actual rgba could represent any image and could have a length of 1 million +.
I've tried various algorithms like this, and converting to a 2d array and then rotating and flattening (which does work) but i'd like to avoid this because speed/memory is an issue.

Body:
Maybe this could be helpful in p5.js for 90 degrees rotation (left and right) (the previous suggested solution seems to have a little 'asymmetric' error):

function rotateRight(img){

  var w = img.width;
  var h = img.height;
  var index, indexr;
  var img2 = createImage(w, h);

  img.loadPixels();
  img2.loadPixels();

  indexr = 0;

  for (let x = 0; x < w; x++) {

    for(let y = h - 1; y >= 0; y--) {

      index = (x+y*w)*4;
      img2.pixels[indexr] = img.pixels[index];
      img2.pixels[indexr + 1] = img.pixels[index+1];
      img2.pixels[indexr + 2] = img.pixels[index+2];
      img2.pixels[indexr + 3] = img.pixels[index+3];

      indexr += 4;

    }
  }

  img.updatePixels();
  img2.updatePixels();

  return img2;

}

function rotateLeft(img){

  var w = img.width;
  var h = img.height;
  var index, indexr;
  var img2 = createImage(w, h);

  img.loadPixels();
  img2.loadPixels();

  indexr = 0;

  for (let x = w - 1; x >= 0; x--) {

    for(let y = 0; y < h; y++) {

      index = (x+y*w)*4;
      img2.pixels[indexr] = img.pixels[index];
      img2.pixels[indexr + 1] = img.pixels[index+1];
      img2.pixels[indexr + 2] = img.pixels[index+2];
      img2.pixels[indexr + 3] = img.pixels[index+3];

      indexr += 4;

    }
  }

  img.updatePixels();
  img2.updatePixels();

  return img2;

}


问:
Is it possible to determine if a named window is open in JavaScript?
I'm working on an inter-site single-sign-on project and have had a pretty little problem dropped in my lap.  When a user logs out of the "parent" site, a particular page needs to be loaded in the popup window containing the "child" site.  However, I can't store a reference to the return value of window.open(…), because the user must be allowed to navigate wherever they like on each site before logging out.

This would be easy if I could assume that the child site is always open, as another window.open(…) to the same named window would change its URL.  However, the popup cannot be caused to appear if it isn't already open (not all users have access to the child site).

I think this gives me two conflicting scenarios.  When the user visits the child site:

User logs in to parent site.

User clicks link for child site

Child site appears in a window named "child_popup".

User browses far and wide in the parent site, forgetting that the child window exists.

User logs out of parent site.

Child popup is redirected to the child site's logout page.

And when the user does not or cannot visit the child site:

User logs in to parent site.

User browses far and wide in the parent site.

User logs out of the parent site.

No popup should appear!

So my limitations are:

I cannot store a reference to the window in JS, as the user may navigate to any number of pages between visiting the child site and logging out.

I have zero control over the contents of the child site (my employer runs the parent site).

The parent site cannot be converted to frames to provide a "global" JS scope. :-)

I was not able to find any relevant resources on Google or SO.  Is there a way to accomplish this?

答:
Use sessions to store the state of the popup.

When the user clicks a link to open the child site, you need do do an  asyncronous javascript call to the parent server that records this user has opened a window to the child site. OR have the link open a page that stores the following session info and returns a Location: header.

(Notice I am using php, $_SESSION[...] is a user specific array of data stored between requests)

```<code>$_SESSION['inChildSite'] = true;
</code>```

When the user logs out of the parent site, this value is checked again, either by an asycnronous javascript call, or by the logout script.

```<code>if ($_SESSION['inChildSite'] ==true )
     echo "<script>window.open(...)</script>"
</code>```

Then display the logout child window. Make sure you unset your session variable when the logout happens.

Voila, profit.

OwnerDisplayName: Ben Blank

ContentLicense: CC BY-SA 2.5

Body:
Use sessions to store the state of the popup.

When the user clicks a link to open the child site, you need do do an  asyncronous javascript call to the parent server that records this user has opened a window to the child site. OR have the link open a page that stores the following session info and returns a Location: header.

(Notice I am using php, $_SESSION[...] is a user specific array of data stored between requests)

$_SESSION['inChildSite'] = true;

When the user logs out of the parent site, this value is checked again, either by an asycnronous javascript call, or by the logout script.

if ($_SESSION['inChildSite'] ==true )
     echo "<script>window.open(...)</script>"

Then display the logout child window. Make sure you unset your session variable when the logout happens.

Voila, profit.

OwnerDisplayName: Byron Whitlock


问:
how to go previous page in asp.net after clicking previous button in asp.net
Hi
I have one label for question, radiobuttonlist for their answers, next button to move next question & previous button. I am displaying one question per page after clicking next button next question appears, but when i click previous button previous question should be display. How i can go to previous question to change answer.

Asp.net c#
Thank you.

答:
One work around is to use History.Back function of Javascript...</p>

```<code>onclick="javascript:History.Back();"
</code>```

ContentLicense: CC BY-SA 3.0

Body:
One work around is to use History.Back function of Javascript...

onclick="javascript:History.Back();"


问:
Gets mismatched input '') as '' expecting
My query in hadoop returns this error

```<code>mismatched input '') as '' expecting ,
</code>```

I don't know why, anybody could find the issue?

```<code>select t1.* , '(sum(t1.'price') as 'price')'
from events t1
join
(
    SELECT min(idsite) as idsite
    FROM events
    GROUP BY idsite
) t2 on t1.idsite = t2.idsite
</code>```

答:
I don't know about hadoop in particular, so maybe there's a dedicated syntax for this, but this looks fishy :

```<code>'(sum(t1.'price') as 'price')'
</code>```

You have quotes pairs enclosed in another quotes pairs. The parser cannot understand this. It will assume the first one to be an opening one, the second one a closing one, the third one to reopen it again, and so on.

Body:
My query in hadoop returns this error

mismatched input '') as '' expecting ,

I don't know why, anybody could find the issue?

select t1.* , '(sum(t1.'price') as 'price')'
from events t1
join
(
    SELECT min(idsite) as idsite
    FROM events
    GROUP BY idsite
) t2 on t1.idsite = t2.idsite

Body:
I don't know about hadoop in particular, so maybe there's a dedicated syntax for this, but this looks fishy :

'(sum(t1.'price') as 'price')'

You have quotes pairs enclosed in another quotes pairs. The parser cannot understand this. It will assume the first one to be an opening one, the second one a closing one, the third one to reopen it again, and so on.


问:
Call Delegate method from UIPopover
I have a popover that gets loaded with a navigation controller, which displays the ItemsView xib, and I have a delegate method so this popover can be dismissed from the main view controller.  This works perfectly fine, until I drill down to the next level in the UITableView (which loads a detail view).  Once the detail view is loaded, I can not call the dismiss method on the main view controller.  I am quite new to delegates and would appreciate any guidance. Thank you!

MainView ---> ItemsView -----> DetailView

How to call delegate method located on MainView from DetailView?

```<code>//Load the popover with first view:

    ItemsView *popoverView = [[ItemsView alloc] init];

UINavigationController *navController = [[UINavigationController alloc] initWithRootViewController:popoverView];

self.popOverController = [[[UIPopoverController alloc] initWithContentViewController:navController] autorelease];

popoverView.delegate = self;

[navController release];

[popOverController setPopoverContentSize:CGSizeMake(450.0, 300.0)];
[popOverController presentPopoverFromRect:addButton.frame inView:self.view permittedArrowDirections:UIPopoverArrowDirectionDown animated:YES];

[popoverView release];
</code>```

Now I push the detail view:

```<code>//Push DetailView

    if(self.secondView == nil) {

    NSLog(@"Called");

    AddDetail *viewController = [[AddDetail alloc] initWithNibName:@"AddDetail" bundle:nil];

    self.secondView = viewController;

    [viewController release];

}

// Setup the animation

secondView.contentSizeForViewInPopover = self.view.bounds.size;
self.contentSizeForViewInPopover = self.view.bounds.size;

[self.navigationController pushViewController:self.secondView animated:YES];

}
</code>```

答:
Write a property and method in Detail view like following.

Property as id myParent

Method as  (void)setParent:parent { myParent = parent;}

while pushing detailviewcontroller call this method as.

[detailViewcontroller setParent:self];

now you can call the method declared in Parent as

[myParent yourMethodName];

Hope this helps

Title: Call Delegate method from UIPopover

Body:
I have a popover that gets loaded with a navigation controller, which displays the ItemsView xib, and I have a delegate method so this popover can be dismissed from the main view controller.  This works perfectly fine, until I drill down to the next level in the UITableView (which loads a detail view).  Once the detail view is loaded, I can not call the dismiss method on the main view controller.  I am quite new to delegates and would appreciate any guidance. Thank you!

MainView ---> ItemsView -----> DetailView

How to call delegate method located on MainView from DetailView?

//Load the popover with first view:

    ItemsView *popoverView = [[ItemsView alloc] init];

UINavigationController *navController = [[UINavigationController alloc] initWithRootViewController:popoverView];

self.popOverController = [[[UIPopoverController alloc] initWithContentViewController:navController] autorelease];

popoverView.delegate = self;

[navController release];

[popOverController setPopoverContentSize:CGSizeMake(450.0, 300.0)];
[popOverController presentPopoverFromRect:addButton.frame inView:self.view permittedArrowDirections:UIPopoverArrowDirectionDown animated:YES];

[popoverView release];

Now I push the detail view:

//Push DetailView

    if(self.secondView == nil) {

    NSLog(@"Called");

    AddDetail *viewController = [[AddDetail alloc] initWithNibName:@"AddDetail" bundle:nil];

    self.secondView = viewController;

    [viewController release];

}

// Setup the animation

secondView.contentSizeForViewInPopover = self.view.bounds.size;
self.contentSizeForViewInPopover = self.view.bounds.size;

[self.navigationController pushViewController:self.secondView animated:YES];

}


问:
What does the expression inside array brackets mean?
I don't understand what this code is trying to say.

expectedContent = content[offset + readOffset : offset + readOffset + readSize]
```

<p>This portion of code is part of a tester, that automatically corrects my homework assignment, and it's inside this function</p>

<pre class="lang-py prettyprint-override"><code> def test_read_section(self, fname):
        score = 0

        subprocess.call(["ipcrm", "shm", self.data["shm_key"]], stderr=open(os.devnull, "w"))
        self.writeString("CREATE_SHM")
        self.writeNumber(int(self.data["shm_size"]))
        r = self.readString()
        if r != "CREATE_SHM":
            return score
        r = self.readString()
        if r != "SUCCESS":
            return score
        # check if the shm actually exists
        shm = self.shmget(int(self.data["shm_key"]), int(self.data["shm_size"]), 0)
        if shm < 0:
            if VERBOSE:
                print "[TESTER] shm with key %s not found" % self.data["shm_key"]
            return score
        shAddr = self.shmat(shm, 0, 0)
        score = 1

        self.writeString("MAP_FILE")
        self.writeString(fname)
        r = self.readString()
        if r != "MAP_FILE":
            return score
        r = self.readString()
        if r != "SUCCESS":
            return score
        score = 2

        sections = getSectionsTable(self.data, fname)
        self.writeString("READ_FROM_FILE_SECTION")
        self.writeNumber(len(sections)+2)
        self.writeNumber(0)
        self.writeNumber(100)
        r = self.readString()
        if r != "READ_FROM_FILE_SECTION":
            return score
        r = self.readString()
        if r != "ERROR":
            return score
        score = 4

        fin = open(fname, "rb")
        content = fin.read()
        fin.close()

        sectIds = random.sample(range(len(sections)), 3)
        for sectId in sectIds:
            _name, _type, offset, size = sections[sectId]
            readOffset = random.randint(0, size/2)
            readSize = random.randint(5, size/2)
            expectedContent = content[offset + readOffset : offset + readOffset + readSize]
            self.writeString("READ_FROM_FILE_SECTION")
            self.writeNumber(sectId+1)
            self.writeNumber(readOffset)
            self.writeNumber(readSize)
            r = self.readString()
            if r != "READ_FROM_FILE_SECTION":
                print "!!!!!%d " % (sectId+1)
                return score
            r = self.readString()
            if r != "SUCCESS":
                print "[ANDREI] sectid:%d, readOffset: %d, readSize: %d" % (sectId+1,readOffset,readSize)
                return score
            readContent = ctypes.string_at(shAddr, readSize)
            if readContent != expectedContent:
                if VERBOSE:
                    print "[TESTER] read content incorrect"
            else:
                score += 2
        return score
</code>```

答: array[x : y] is a sub array which contains all elements between index x and index y - 1.

Body:
I don't understand what this code is trying to say.

expectedContent = content[offset + readOffset : offset + readOffset + readSize]

This portion of code is part of a tester, that automatically corrects my homework assignment, and it's inside this function

 def test_read_section(self, fname):
        score = 0

        subprocess.call(["ipcrm", "shm", self.data["shm_key"]], stderr=open(os.devnull, "w"))
        self.writeString("CREATE_SHM")
        self.writeNumber(int(self.data["shm_size"]))
        r = self.readString()
        if r != "CREATE_SHM":
            return score
        r = self.readString()
        if r != "SUCCESS":
            return score
        # check if the shm actually exists
        shm = self.shmget(int(self.data["shm_key"]), int(self.data["shm_size"]), 0)
        if shm < 0:
            if VERBOSE:
                print "[TESTER] shm with key %s not found" % self.data["shm_key"]
            return score
        shAddr = self.shmat(shm, 0, 0)
        score = 1

        self.writeString("MAP_FILE")
        self.writeString(fname)
        r = self.readString()
        if r != "MAP_FILE":
            return score
        r = self.readString()
        if r != "SUCCESS":
            return score
        score = 2

        sections = getSectionsTable(self.data, fname)
        self.writeString("READ_FROM_FILE_SECTION")
        self.writeNumber(len(sections)+2)
        self.writeNumber(0)
        self.writeNumber(100)
        r = self.readString()
        if r != "READ_FROM_FILE_SECTION":
            return score
        r = self.readString()
        if r != "ERROR":
            return score
        score = 4

        fin = open(fname, "rb")
        content = fin.read()
        fin.close()

        sectIds = random.sample(range(len(sections)), 3)
        for sectId in sectIds:
            _name, _type, offset, size = sections[sectId]
            readOffset = random.randint(0, size/2)
            readSize = random.randint(5, size/2)
            expectedContent = content[offset + readOffset : offset + readOffset + readSize]
            self.writeString("READ_FROM_FILE_SECTION")
            self.writeNumber(sectId+1)
            self.writeNumber(readOffset)
            self.writeNumber(readSize)
            r = self.readString()
            if r != "READ_FROM_FILE_SECTION":
                print "!!!!!%d " % (sectId+1)
                return score
            r = self.readString()
            if r != "SUCCESS":
                print "[ANDREI] sectid:%d, readOffset: %d, readSize: %d" % (sectId+1,readOffset,readSize)
                return score
            readContent = ctypes.string_at(shAddr, readSize)
            if readContent != expectedContent:
                if VERBOSE:
                    print "[TESTER] read content incorrect"
            else:
                score += 2
        return score


问:
how to get good color separation for a few Matlab plot traces?
I need very distinct separation between several plot() traces. several means perhaps 2 or 3 or 5 time history traces or XY plots.

prism is nice with 7 distinctly separate colors but its yellow is hardly visible on the default white background.

Here's an example with default colors. They are quite nice but still somewhat muted pastels. I'd prefer brighter colors than default:

```<code>figure(1)
clf
x=1:100;
for i = 1:7
  y=i*log(x);
  plot(x,y, 'Linewidth', 5);
  legendStr{i}=sprintf('line %i',i);
  hold on
end
grid on
legend(legendStr,'location','best')
</code>```

how can I use prism or some other colormap without bright yellow? is there a better approach for plotting a few separate traces in the same figure with really clear color separation?

答:
The prism palette has 6 unique colors, not 7. cm=prism(6) returns these 6 colors. The 3rd one is the yellow, we can remove it with cm(3,:)=[].

We can now set the default color order for plots using the ColorOrder property of the axes object (see here):

```<code>cm = prism(6);
cm(3,:) = [];
set(gca,'ColorOrder',cm);
</code>```

Now, when plotting to this axes object, those 5 colors will be cycled through (remember to set hold on so the axes properties are not reset).

To change the colors for all subsequent plots in the current MATLAB session, change the default color order:

```<code>set(groot,'defaultAxesColorOrder',cm)
</code>```

Body:
I need very distinct separation between several plot() traces. several means perhaps 2 or 3 or 5 time history traces or XY plots.

prism is nice with 7 distinctly separate colors but its yellow is hardly visible on the default white background.

Here's an example with default colors. They are quite nice but still somewhat muted pastels. I'd prefer brighter colors than default:

figure(1)
clf
x=1:100;
for i = 1:7
  y=i*log(x);
  plot(x,y, 'Linewidth', 5);
  legendStr{i}=sprintf('line %i',i);
  hold on
end
grid on
legend(legendStr,'location','best')

how can I use prism or some other colormap without bright yellow? is there a better approach for plotting a few separate traces in the same figure with really clear color separation?

Body:
The prism palette has 6 unique colors, not 7. cm=prism(6) returns these 6 colors. The 3rd one is the yellow, we can remove it with cm(3,:)=[].

We can now set the default color order for plots using the ColorOrder property of the axes object (see here):

cm = prism(6);
cm(3,:) = [];
set(gca,'ColorOrder',cm);

Now, when plotting to this axes object, those 5 colors will be cycled through (remember to set hold on so the axes properties are not reset).

To change the colors for all subsequent plots in the current MATLAB session, change the default color order:

set(groot,'defaultAxesColorOrder',cm)


问:
Sum old variable with new variable
I have an Adapter that points to items that have a "Time" attribute.
If the user taps a button on one item and on another I'd need both Time values to be summed. I can't achieve this.

I've tried stuff like time += time, separate those values etc

This was the last thing I was trying. Yeah it makes no sense but idk, nothing had worked.

```<code> [...]
  add.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                GetTime(gameItem.getTime());

            }
        });
    }

    private int GetTime(int timeplayed) {
        time += timeplayed;

        return time;
    }
</code>```

Which should have been like at the start there is an empty variable time.
timeplayed is added to it and is returned as time.
So eveytime you tap the button the timeplayed gets added and returned as time

Actually tho, instead of summing the old variable with the new one it either does just a 0 + number or it sums the new variable with itself...

答:
From what you said it sounds to me that the field value is being altered or actually not initialized at all before you get to actually do your mathematical operations on it. So the answer to your question is simple: make sure the value of the field is not changed before your code reaches the private int GetTime(int timeplayed) method.

In any case your code absolutely should work as the syntax of the operation is correct. time += timeplayed is equivalent to time = time + timeplayed. I can't see the rest of your code so I can't really say whats going on underneath the hood but if you are willing to share the rest of your code, pretty much anywhere the time variable is being altered (including the place where it's being actually initialized) I could help you out further.

But first you need to make sure the variable has an expected value (not the same as timeplayed and not 0). You can do this through debugging either with using breakpoints or logging the value in console before the operation gets executed.

Hope this helps, let me know how it goes.

Title: Sum old variable with new variable

Body:
I have an Adapter that points to items that have a "Time" attribute.
If the user taps a button on one item and on another I'd need both Time values to be summed. I can't achieve this.

I've tried stuff like time += time, separate those values etc

This was the last thing I was trying. Yeah it makes no sense but idk, nothing had worked.

 [...]
  add.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                GetTime(gameItem.getTime());

            }
        });
    }

    private int GetTime(int timeplayed) {
        time += timeplayed;

        return time;
    }

Which should have been like at the start there is an empty variable time.
timeplayed is added to it and is returned as time.
So eveytime you tap the button the timeplayed gets added and returned as time

Actually tho, instead of summing the old variable with the new one it either does just a 0 + number or it sums the new variable with itself...


答: Actually this is not an operation, but a truncation  of the list 'content', starting from index: (offset + readOffset),  and ending at inex: (offset + readOffset + readSize)</p>

Title: What does the expression inside array brackets mean?


问:
What is the purpose of Encapsulation? Does it provide a security layer?
As much as I have studied about encapsulation it is about hiding data to prevent from manipulation. For that we declare the private variables so that couldn't be accessible from out of class. But we can implement or access them out of class using setter getter method. So if we have to implement it using setter getter method than any other person can implement it using setter getter method. . So how we are safe from manipulation??</p>

答:
Encapsulation isn't a security measure in that it prevents people from messing with your code. It's a security measure in the sense that people can't go directly in and change variables without going through your proper channels. Consider the following psuedocode.

```<code>class ProgressBar {
    public int maximum;
    public int current;
}
</code>```

vs

```<code>class ProgressBar {
    private int maximum;
    private int current;

    ...
    public set_current(int amount) {
        if (amount <= this.maximum) this.current = amount;
    }
}
</code>```

In the top example, users could go in and mess with current and break the
progress bar. In the bottom example, your setter protects from that.

Body:
Encapsulation isn't a security measure in that it prevents people from messing with your code. It's a security measure in the sense that people can't go directly in and change variables without going through your proper channels. Consider the following psuedocode.

class ProgressBar {
    public int maximum;
    public int current;
}

vs

class ProgressBar {
    private int maximum;
    private int current;

    ...
    public set_current(int amount) {
        if (amount <= this.maximum) this.current = amount;
    }
}

In the top example, users could go in and mess with current and break the
progress bar. In the bottom example, your setter protects from that.


问:
problem in writing a character in *.TXT file in MQL
I'm trying to to read from a TXT file and do some calculation and write it back to another TXT file but when I read the character it changes to ASCII number (ex : '1' convert to 50) and when I try to write it in another file it's the ASCII number. How can I change it to that character I want?

int wf=FileOpen("wf.txt",FILE_WRITE|FILE_ANSI|FILE_TXT);
int rf=FileOpen("rf.txt",FILE_READ|FILE_ANSI|FILE_TXT);
str_size=FileReadInteger(rf,INT_VALUE);  //the TXT I read is 1234
str=FileReadString(rf,str_size);
StringToCharArray(str,data1,0,StringLen(str));
RandonNum[0]= str[1];
RandonNum[1]= str[2];
RandonNum[2]= str[3];
FileWrite(wf,str[1],str[2],str[3]);     //the TXT I write is 505152
```

答:
FileReadInteger() is reserved for binary type files.

Unfortunately this is not explicitly stated in the documentation.

Use FileReadNumber() to read a number from txt file. It will return the number as a double, but it can be cast to an integer using a type cast (int)double_value.


问:
Add Start and Stop point parameters to method that adds sums of numbers in a string
I am trying to modify a method from the post marked as duplicate that will allow me to enter a string containing numbers, like "Here are77 with88 monsters" which I adds the sum of the numbers in the string to return 14 and 16 if I evaluated the entire string. How would I enhance this method to evaluate start and stop points in the string? So if I wanted to evaluate position 3 through 10, it would return just 14, since 77 are numbers between positions 3 and 10?

I have done quite a bit of research, but I'm really new to Java, and don't know enough yet to figure this out. The code below is where I'm at now:

```<code>    String s = "Here are77 with 88 monsters";
    String[] st = s.split(" ");
    for (int i=0 ; i<st.length ; i++) {
        if (st[i].matches("[0-9]+")) {
            String[] c = st[i].split("(?=\\d)");
            int sum = 0;
            for(int j=0 ; j<c.length ; j++) {
                sum += Integer.parseInt(c[j]);
            }
          return sum;}
        }
</code>```

I need help to add the two extra position parameters so I can use this code above to calculate the sum of the digits within those margins in a method.

答:
Your actual code doesn't work well, and the possible duplicate is far too complicated.

Sum digits in a String

remove all chars that are not digits

split them individually

parse and sum them

```<code>static int sumstring(String s) {
    String[] st = s.replaceAll("\\D", "").split("");
    int sum = 0;
    for (String digit : st) {
        sum += Integer.parseInt(digit);
    }
    return sum;
}
</code>```

Sum digits in a String, with start and stop

way 1. and add substring() method

```<code>static int sumstring(String s, int start, int stop) {
    s = s.substring(start, stop);
    String[] st = s.replaceAll("\\D", "").split("");
    int sum = 0;
    for (String digit : st) {
        sum += Integer.parseInt(digit);
    }
    return sum;
}
</code>```

To get an array of the sums (each number):

```<code>static int[] sumstring(String s) {
    String[] st = s.replaceAll("\\D", " ").trim().split("\\s+");
    System.out.println(Arrays.toString(st));
    int[] sums = new int[st.length];
    for (int i = 0; i < sums.length; i++) {
        int sum = 0;
        for (String digit : st[i].split("")) {
            sum += Integer.parseInt(digit);
        }
        sums[i] = sum;
    }
    return sums;
}
</code>```

Replace number bu sum of its digits

```<code>static String sumstring(String s) {
    String[] st = s.split("\\s+");
    for (int i = 0; i < st.length; i++) {
        String word = st[i];
        if (word.matches("^[0-9]+$")) {
            int sum = 0;
            for (String digit : word.split("")) {
                sum += Integer.parseInt(digit);
            }
            st[i] = "" + sum;
        }
    }
    return String.join(" ", st);
}
</code>```

Body:
I am trying to modify a method from the post marked as duplicate that will allow me to enter a string containing numbers, like "Here are77 with88 monsters" which I adds the sum of the numbers in the string to return 14 and 16 if I evaluated the entire string. How would I enhance this method to evaluate start and stop points in the string? So if I wanted to evaluate position 3 through 10, it would return just 14, since 77 are numbers between positions 3 and 10?

I have done quite a bit of research, but I'm really new to Java, and don't know enough yet to figure this out. The code below is where I'm at now:

    String s = "Here are77 with 88 monsters";
    String[] st = s.split(" ");
    for (int i=0 ; i<st.length ; i++) {
        if (st[i].matches("[0-9]+")) {
            String[] c = st[i].split("(?=\\d)");
            int sum = 0;
            for(int j=0 ; j<c.length ; j++) {
                sum += Integer.parseInt(c[j]);
            }
          return sum;}
        }

I need help to add the two extra position parameters so I can use this code above to calculate the sum of the digits within those margins in a method.

Body:
Your actual code doesn't work well, and the possible duplicate is far too complicated.

Sum digits in a String

remove all chars that are not digits

split them individually

parse and sum them

static int sumstring(String s) {
    String[] st = s.replaceAll("\\D", "").split("");
    int sum = 0;
    for (String digit : st) {
        sum += Integer.parseInt(digit);
    }
    return sum;
}

Sum digits in a String, with start and stop

way 1. and add substring() method

static int sumstring(String s, int start, int stop) {
    s = s.substring(start, stop);
    String[] st = s.replaceAll("\\D", "").split("");
    int sum = 0;
    for (String digit : st) {
        sum += Integer.parseInt(digit);
    }
    return sum;
}

To get an array of the sums (each number):

static int[] sumstring(String s) {
    String[] st = s.replaceAll("\\D", " ").trim().split("\\s+");
    System.out.println(Arrays.toString(st));
    int[] sums = new int[st.length];
    for (int i = 0; i < sums.length; i++) {
        int sum = 0;
        for (String digit : st[i].split("")) {
            sum += Integer.parseInt(digit);
        }
        sums[i] = sum;
    }
    return sums;
}

Replace number bu sum of its digits

static String sumstring(String s) {
    String[] st = s.split("\\s+");
    for (int i = 0; i < st.length; i++) {
        String word = st[i];
        if (word.matches("^[0-9]+$")) {
            int sum = 0;
            for (String digit : word.split("")) {
                sum += Integer.parseInt(digit);
            }
            st[i] = "" + sum;
        }
    }
    return String.join(" ", st);
}


问:
Rank Pandas dataframe by quantile
I have a Pandas dataframe in which each column represents a separate property, and each row holds the properties' value on a specific date:

```<code>import pandas as pd

dfstr = \
'''         AC        BO         C       CCM        CL       CRD        CT        DA        GC        GF
2010-01-19  0.844135 -0.194530 -0.231046  0.245615 -0.581238 -0.593562  0.057288  0.655903  0.823997  0.221920
2010-01-20 -0.204845 -0.225876  0.835611 -0.594950 -0.607364  0.042603  0.639168  0.816524  0.210653  0.237833
2010-01-21  0.824852 -0.216449 -0.220136  0.234343 -0.611756 -0.624060  0.028295  0.622516  0.811741  0.201083'''
df = pd.read_csv(pd.compat.StringIO(dfstr), sep='\s+')
</code>```

Using the rank method, I can find the percentile rank of each property with respect to a specific date:

```<code>df.rank(axis=1, pct=True)
</code>```

Output:

```<code>             AC   BO    C  CCM   CL  CRD   CT   DA   GC   GF
2010-01-19  1.0  0.4  0.3  0.7  0.2  0.1  0.5  0.8  0.9  0.6
2010-01-20  0.4  0.3  1.0  0.2  0.1  0.5  0.8  0.9  0.6  0.7
2010-01-21  1.0  0.4  0.3  0.7  0.2  0.1  0.5  0.8  0.9  0.6
</code>```

What I'd like to get instead is the quantile (eg quartile, quintile, decile, etc) rank of each property. For example, for quintile rank my desired output would be:

```<code>             AC   BO    C  CCM   CL  CRD   CT   DA   GC   GF
2010-01-19   5    2     2  4     1   1     3    4    5    3
2010-01-20   2    2     5  1     1   3     4    5    3    4
2010-01-21   5    2     2  4     1   1     3    4    5    3
</code>```

I might be missing something, but there doesn't seem to a built-in way to do this kind of quantile ranking with Pandas. What's the simplest way to get my desired output?

答:
Method 1 mul & np.ceil

You were quite close with the rank. Just multiplying by 5 with .mul to get the desired quantile, also rounding up with np.ceil:

```<code>np.ceil(df.rank(axis=1, pct=True).mul(5))
</code>```

Output

```<code>             AC   BO    C  CCM   CL  CRD   CT   DA   GC   GF
2010-01-19  5.0  2.0  2.0  4.0  1.0  1.0  3.0  4.0  5.0  3.0
2010-01-20  2.0  2.0  5.0  1.0  1.0  3.0  4.0  5.0  3.0  4.0
2010-01-21  5.0  2.0  2.0  4.0  1.0  1.0  3.0  4.0  5.0  3.0
</code>```

If you want integers use astype:

```<code>np.ceil(df.rank(axis=1, pct=True).mul(5)).astype(int)
</code>```

Or even better
Since pandas version 0.24.0 we have nullable integer type: Int64.

So we can use :

```<code>np.ceil(df.rank(axis=1, pct=True).mul(5)).astype('Int64')
</code>```

Output

```<code>            AC  BO  C  CCM  CL  CRD  CT  DA  GC  GF
2010-01-19   5   2  2    4   1    1   3   4   5   3
2010-01-20   2   2  5    1   1    3   4   5   3   4
2010-01-21   5   2  2    4   1    1   3   4   5   3
</code>```

Method 2 scipy.stats.percentileofscore

```<code>d = df.apply(lambda x: [np.ceil(stats.percentileofscore(x, a, 'rank')*0.05) for a in x], axis=1).values

pd.DataFrame(data=np.concatenate(d).reshape(d.shape[0], len(d[0])),
             columns=df.columns,
             dtype='int',
             index=df.index)
</code>```

Output

```<code>            AC  BO  C  CCM  CL  CRD  CT  DA  GC  GF
2010-01-19   5   2  2    4   1    1   3   4   5   3
2010-01-20   2   2  5    1   1    3   4   5   3   4
2010-01-21   5   2  2    4   1    1   3   4   5   3
</code>```

Title: Rank Pandas dataframe by quantile

Body:
I have a Pandas dataframe in which each column represents a separate property, and each row holds the properties' value on a specific date:

import pandas as pd

dfstr = \
'''         AC        BO         C       CCM        CL       CRD        CT        DA        GC        GF
2010-01-19  0.844135 -0.194530 -0.231046  0.245615 -0.581238 -0.593562  0.057288  0.655903  0.823997  0.221920
2010-01-20 -0.204845 -0.225876  0.835611 -0.594950 -0.607364  0.042603  0.639168  0.816524  0.210653  0.237833
2010-01-21  0.824852 -0.216449 -0.220136  0.234343 -0.611756 -0.624060  0.028295  0.622516  0.811741  0.201083'''
df = pd.read_csv(pd.compat.StringIO(dfstr), sep='\s+')

Using the rank method, I can find the percentile rank of each property with respect to a specific date:

df.rank(axis=1, pct=True)

Output:

             AC   BO    C  CCM   CL  CRD   CT   DA   GC   GF
2010-01-19  1.0  0.4  0.3  0.7  0.2  0.1  0.5  0.8  0.9  0.6
2010-01-20  0.4  0.3  1.0  0.2  0.1  0.5  0.8  0.9  0.6  0.7
2010-01-21  1.0  0.4  0.3  0.7  0.2  0.1  0.5  0.8  0.9  0.6

What I'd like to get instead is the quantile (eg quartile, quintile, decile, etc) rank of each property. For example, for quintile rank my desired output would be:

             AC   BO    C  CCM   CL  CRD   CT   DA   GC   GF
2010-01-19   5    2     2  4     1   1     3    4    5    3
2010-01-20   2    2     5  1     1   3     4    5    3    4
2010-01-21   5    2     2  4     1   1     3    4    5    3

I might be missing something, but there doesn't seem to a built-in way to do this kind of quantile ranking with Pandas. What's the simplest way to get my desired output?

Body:
Method 1 mul & np.ceil

You were quite close with the rank. Just multiplying by 5 with .mul to get the desired quantile, also rounding up with np.ceil:

np.ceil(df.rank(axis=1, pct=True).mul(5))

Output

             AC   BO    C  CCM   CL  CRD   CT   DA   GC   GF
2010-01-19  5.0  2.0  2.0  4.0  1.0  1.0  3.0  4.0  5.0  3.0
2010-01-20  2.0  2.0  5.0  1.0  1.0  3.0  4.0  5.0  3.0  4.0
2010-01-21  5.0  2.0  2.0  4.0  1.0  1.0  3.0  4.0  5.0  3.0

If you want integers use astype:

np.ceil(df.rank(axis=1, pct=True).mul(5)).astype(int)

Or even better
Since pandas version 0.24.0 we have nullable integer type: Int64.

So we can use :

np.ceil(df.rank(axis=1, pct=True).mul(5)).astype('Int64')

Output

            AC  BO  C  CCM  CL  CRD  CT  DA  GC  GF
2010-01-19   5   2  2    4   1    1   3   4   5   3
2010-01-20   2   2  5    1   1    3   4   5   3   4
2010-01-21   5   2  2    4   1    1   3   4   5   3

Method 2 scipy.stats.percentileofscore

d = df.apply(lambda x: [np.ceil(stats.percentileofscore(x, a, 'rank')*0.05) for a in x], axis=1).values

pd.DataFrame(data=np.concatenate(d).reshape(d.shape[0], len(d[0])),
             columns=df.columns,
             dtype='int',
             index=df.index)

Output

            AC  BO  C  CCM  CL  CRD  CT  DA  GC  GF
2010-01-19   5   2  2    4   1    1   3   4   5   3
2010-01-20   2   2  5    1   1    3   4   5   3   4
2010-01-21   5   2  2    4   1    1   3   4   5   3


问:
Looking for example of how to display text in an NSWindow programmatically?
I am trying to display text in a HUD style semi-transparent window. I have got the code for the window all set up, but I have been scouring the documents for two days now and haven't come up with a way that works to actually display text in the window. Since I am using AppleScriptObjC in Script Debugger, and not Xcode, I'd rather do this programmatically and not have to switch to Xcode and use IB. (I did spend some time messing around with IB, but it is not very intuitive to be honest. So I thought I would check this form before going through the guides on how to get started using IB).

So I was given some advice to "create an NSTextField and add it to your window's contentView". So I have tried many different set ups of trying to init a NSTextField (And NSTextView), and I may have been able to get that part correct, but getting the text to actually display in the window has been a bigger challenge than I expected. I have included a code snippit of the code I am using to generate the window.

```<code>tell (NSWindow's alloc()'s ¬
            initWithContentRect:{{theWidth, theHeight}, {640, 480}} ¬
                styleMask:NSBorderlessWindowMask ¬
                backing:NSBackingStoreBuffered ¬
                defer:true)

            setOpaque_(yes)
            setAlphaValue_(0.5)
            setBackgroundColor_(NSColor's grayColor())
            setReleasedWhenClosed_(yes)
            setExcludedFromWindowsMenu_(yes)
            orderFrontRegardless()
            delay 1
            |close|()
        end tell
</code>```

My hope is to be able to get an NSText View in that Window in order to display some text in it. So far I haven't come close. The errors I generally get are about "unrecognized selector sent to instance". So it is pretty obvious that I am doing something wrong. I hope there is an easy way to accomplish this that I haven't yet come across.

答:
It sounds like there isn't a target for some of the methods.  In a tell statement, usually the target is implied, but sometimes AppleScript can't figure it out.  You also don't get the newer method syntax, so I've had better luck just specifying the target for everything - also note that object properties can usually be set directly instead of using their setter method.

I can't tell from your snippet, but you also need to add the textView to the window's contentView, for example:

```<code>use AppleScript version "2.4" -- Yosemite (10.10) or later
use framework "Foundation"
use scripting additions

# create a text view
tell (current application's NSTextView's alloc's initWithFrame:{{20, 20}, {600, 440}})
  set theTextView to it
end tell

# create a window
tell (current application's NSWindow's alloc()'s ¬
    initWithContentRect:{{200, 600}, {640, 480}} ¬
    styleMask:(current application's NSBorderlessWindowMask) ¬
    backing:(current application's NSBackingStoreBuffered) ¬
    defer:true)
  set theWindow to it
  set its opaque to true
  set its alphaValue to 0.5
  set its backgroundColor to (current application's NSColor's grayColor)
  set its releasedWhenClosed to true
  set its excludedFromWindowsMenu to true
end tell

theWindow's contentView's addSubview:theTextView
theTextView's setString:"this is a test"
theWindow's orderFrontRegardless()
delay 5
theWindow's |close|()
</code>```

Body:
I am trying to display text in a HUD style semi-transparent window. I have got the code for the window all set up, but I have been scouring the documents for two days now and haven't come up with a way that works to actually display text in the window. Since I am using AppleScriptObjC in Script Debugger, and not Xcode, I'd rather do this programmatically and not have to switch to Xcode and use IB. (I did spend some time messing around with IB, but it is not very intuitive to be honest. So I thought I would check this form before going through the guides on how to get started using IB).

So I was given some advice to "create an NSTextField and add it to your window's contentView". So I have tried many different set ups of trying to init a NSTextField (And NSTextView), and I may have been able to get that part correct, but getting the text to actually display in the window has been a bigger challenge than I expected. I have included a code snippit of the code I am using to generate the window.

tell (NSWindow's alloc()'s ¬
            initWithContentRect:{{theWidth, theHeight}, {640, 480}} ¬
                styleMask:NSBorderlessWindowMask ¬
                backing:NSBackingStoreBuffered ¬
                defer:true)

            setOpaque_(yes)
            setAlphaValue_(0.5)
            setBackgroundColor_(NSColor's grayColor())
            setReleasedWhenClosed_(yes)
            setExcludedFromWindowsMenu_(yes)
            orderFrontRegardless()
            delay 1
            |close|()
        end tell

My hope is to be able to get an NSText View in that Window in order to display some text in it. So far I haven't come close. The errors I generally get are about "unrecognized selector sent to instance". So it is pretty obvious that I am doing something wrong. I hope there is an easy way to accomplish this that I haven't yet come across.

Body:
It sounds like there isn't a target for some of the methods.  In a tell statement, usually the target is implied, but sometimes AppleScript can't figure it out.  You also don't get the newer method syntax, so I've had better luck just specifying the target for everything - also note that object properties can usually be set directly instead of using their setter method.

I can't tell from your snippet, but you also need to add the textView to the window's contentView, for example:

use AppleScript version "2.4" -- Yosemite (10.10) or later
use framework "Foundation"
use scripting additions

# create a text view
tell (current application's NSTextView's alloc's initWithFrame:{{20, 20}, {600, 440}})
  set theTextView to it
end tell

# create a window
tell (current application's NSWindow's alloc()'s ¬
    initWithContentRect:{{200, 600}, {640, 480}} ¬
    styleMask:(current application's NSBorderlessWindowMask) ¬
    backing:(current application's NSBackingStoreBuffered) ¬
    defer:true)
  set theWindow to it
  set its opaque to true
  set its alphaValue to 0.5
  set its backgroundColor to (current application's NSColor's grayColor)
  set its releasedWhenClosed to true
  set its excludedFromWindowsMenu to true
end tell

theWindow's contentView's addSubview:theTextView
theTextView's setString:"this is a test"
theWindow's orderFrontRegardless()
delay 5
theWindow's |close|()


问:
Is this correct way of checking SHA256 on Mac?

I've downloaded the tar file for OpenJDK 12 for macOS from here.

The SHA256 for the same provided here suggests it should be:

cba6f42f82496f62c51fb544e243d440984d442bdc906550a30428d8be6189e5
```

<p>However when I run the checksum on my laptop it comes out different.</p>

<pre class="lang-none prettyprint-override"><code>$ gsha256sum openjdk-12.0.1_osx-x64_bin.tar
dba2de6203b798f1931c1158e0eb89227ef855a6f4515d8d5a7d2e3a9f8c18bb  openjdk-12.0.1_osx-x64_bin.tar
$ openssl dgst -sha256 openjdk-12.0.1_osx-x64_bin.tar
SHA256(openjdk-12.0.1_osx-x64_bin.tar)= dba2de6203b798f1931c1158e0eb89227ef855a6f4515d8d5a7d2e3a9f8c18b
</code>```

Am I incorrectly checking the checksum?

If so, what is the right way of doing so, and is this tar file safe to install on my laptop?

答:
From your post it looks like you're comparing the ungzipped .tar file, which your browser/os might have 'helpfully' unpacked for you after the download. You need to compute the checksum on the .tar.gz file.

When I run the command on the .tar.gz file, the checksum comes out as expected.

Body:
I've downloaded the tar file for OpenJDK 12 for macOS from here.

The SHA256 for the same provided here suggests it should be:

cba6f42f82496f62c51fb544e243d440984d442bdc906550a30428d8be6189e5

However when I run the checksum on my laptop it comes out different.

$ gsha256sum openjdk-12.0.1_osx-x64_bin.tar
dba2de6203b798f1931c1158e0eb89227ef855a6f4515d8d5a7d2e3a9f8c18bb  openjdk-12.0.1_osx-x64_bin.tar
$ openssl dgst -sha256 openjdk-12.0.1_osx-x64_bin.tar
SHA256(openjdk-12.0.1_osx-x64_bin.tar)= dba2de6203b798f1931c1158e0eb89227ef855a6f4515d8d5a7d2e3a9f8c18b

Am I incorrectly checking the checksum?

If so, what is the right way of doing so, and is this tar file safe to install on my laptop?


问:
Dependent Job doesn't get triggered when SNAPSHOT is built
I have a project (B,C,D) that depends on a Common-project(A).
When A is built B,C,D should be triggered using pipelines.
All builds are successful.

Project-A is producing a JAR, which is not published anywhere else than locally (~/.m2/...)

I normally use Jenkinsfiles, but for now I've also setup minimal pipeline-jobs with the code directly in Jenkins while trying to get the triggering to work.

Below is the minimal pipeline jobs I've created for testing.

For all projects I've ticked "Build whenever a SNAPSHOT dependency is built"

In the build logs I can see this for both projects:
[INFO] [jenkins-event-spy] Generated /var/lib/jenkins/workspace/ProjectName@tmp/withMavend4286f3f/maven-spy-20190411-121907-5107444902202263660415.log

Project A pipeline

```<code>node{
  stage ('Build'){
      git branch: 'master', credentialsId: 'theID', url: 'git@github.com:ORG/PROJECT-A.git'
     withMaven(
        maven: 'Maven_3_6_0',
        jdk: 'Java1.8'

        ) {
          sh "mvn clean install"
    }
  }
}
</code>```

Project B pipeline

```<code>node{
    stage ('Build'){
      git branch: 'master', credentialsId: 'theID', url: 'git@github.com:ORG/PROJECT-B.git'
     withMaven(
        maven: 'Maven_3_6_0',
        jdk: 'Java1.8'
        ) {
            sh "mvn clean install"
        }
    }
}
</code>```

Project B pom.xml dependency declaration

```<code>        <dependency>
            <groupId>com.myorg</groupId>
            <artifactId>projecta</artifactId>
            <version>1.0-SNAPSHOT</version>
        </dependency>
</code>```

I cannot figure out what else is needed to get project B triggered whenever A is built.

答:
Please see the Pipeline Maven Plugin FAQ > My downstream pipelines don't get triggered even if I use "Build whenever a SNAPSHOT dependency is built"

To configure the Jenkins Pipeline Maven integration to trigger downstream pipelines on mvn install:

Either change the global configuration:

Navigate to "Manage Jenkins / Global Tools Configuration"

Click on "Add Publisher" and select "Pipeline Graph Publisher"

In the section "Downstream Pipeline Triggers", change the the "Maven lifecycle threshold" from "deploy" to "install"
 Global Configuration > Trigger Downstream Pipelines > Maven lifecycle threshold">

Or in the script of the upstream pipeline, configure withMaven(options: [pipelineGraphPublisher(lifecycleThreshold: 'install')]) {...}

Please also don't forget on the downstream pipelines (project B), to enable "Build whenever a SNAPSHOT dependency is built" (properties([pipelineTriggers([snapshotDependencies()])]))

Body:
I have a project (B,C,D) that depends on a Common-project(A).
When A is built B,C,D should be triggered using pipelines.
All builds are successful.

Project-A is producing a JAR, which is not published anywhere else than locally (~/.m2/...)

I normally use Jenkinsfiles, but for now I've also setup minimal pipeline-jobs with the code directly in Jenkins while trying to get the triggering to work.

Below is the minimal pipeline jobs I've created for testing.

For all projects I've ticked "Build whenever a SNAPSHOT dependency is built"

In the build logs I can see this for both projects:
[INFO] [jenkins-event-spy] Generated /var/lib/jenkins/workspace/ProjectName@tmp/withMavend4286f3f/maven-spy-20190411-121907-5107444902202263660415.log

Project A pipeline

node{
  stage ('Build'){
      git branch: 'master', credentialsId: 'theID', url: 'git@github.com:ORG/PROJECT-A.git'
     withMaven(
        maven: 'Maven_3_6_0',
        jdk: 'Java1.8'

        ) {
          sh "mvn clean install"
    }
  }
}

Project B pipeline

node{
    stage ('Build'){
      git branch: 'master', credentialsId: 'theID', url: 'git@github.com:ORG/PROJECT-B.git'
     withMaven(
        maven: 'Maven_3_6_0',
        jdk: 'Java1.8'
        ) {
            sh "mvn clean install"
        }
    }
}

Project B pom.xml dependency declaration

        <dependency>
            <groupId>com.myorg</groupId>
            <artifactId>projecta</artifactId>
            <version>1.0-SNAPSHOT</version>
        </dependency>

I cannot figure out what else is needed to get project B triggered whenever A is built.

Body:
Please see the Pipeline Maven Plugin FAQ > My downstream pipelines don't get triggered even if I use "Build whenever a SNAPSHOT dependency is built"

To configure the Jenkins Pipeline Maven integration to trigger downstream pipelines on mvn install:

Either change the global configuration:

Navigate to "Manage Jenkins / Global Tools Configuration"

Click on "Add Publisher" and select "Pipeline Graph Publisher"

In the section "Downstream Pipeline Triggers", change the the "Maven lifecycle threshold" from "deploy" to "install"

Or in the script of the upstream pipeline, configure withMaven(options: [pipelineGraphPublisher(lifecycleThreshold: 'install')]) {...}

Please also don't forget on the downstream pipelines (project B), to enable "Build whenever a SNAPSHOT dependency is built" (properties([pipelineTriggers([snapshotDependencies()])]))


问:
Create ES6 module 'equivalent' using webpack output.target options
Firstly, apologies! I struggled summing up the issue in the title so please feel free to amend.

The problem

Suppose I have two ES6 files that expose default functions

```<code>// file_1.js
export default function(){ /* do the thing */ }

// file_2.js
export default function(){ /* do the other thing */ }
</code>```

Now I bundle file_1 into a module using webpack (w/ babel loader) using the following output configuration

```<code>// webpack config 1.
{
   ...
   entry : '/path/to/file_1.js'
   output : {
      path: '/where/it/goes/',
      filename : 'main.js',
      library: 'my_component',
      libraryTarget: 'var'
   }
}
</code>```

I also have a minimal package.json so it can be imported as a npm module { name: 'file_1', main: 'main.js' }

Now the challenge comes when I want to bundle together the code from file_1 and the module file_2 and a uniform manner.

```<code>// master_entry.js

const components = {
    file_1 : require('file_1'),
    file_2 : require('/path/to/file_2')
}
</code>```

If I bundle the above and look at the resulting form of components it looks like so

```<code>console.log(components.file_1)
// outputs
Module {__esModule: true, Symbol(Symbol.toStringTag): "Module" }
</code>```

```<code>console.log(components.file_2)
// outputs
Module {default: f, __esModule: true, Symbol(Symbol.toStringTag): "Module" }
</code>```

Hence for file_2 (which was not prebundled) I have the default function available - which is what I want. How do I achieve the same thing when bundling file_1?

I have tried playing around with the webpack output options such as library, libraryTarget, libraryExport. However I am a bit lost and have spent far to long in the docs now - hence the cry for help!

Thanks in advance.

For clarity

file_1.js -webpack-> package file_1 (entry point file_1.js)
master_entry -webpack-> main.js

I.e, webpack is running first on file_1.js and subsequently upon the combination of importing the file_1 package and file_2.js.

答:
I think I have a solution ;)

```<code>// file_1.js
export default function file1(){ console.log('file_1') }

// file_2.js
export default function file2(){ console.log('file_2') }
</code>```

The webpack.config.js should look like this

```<code>entry: {
  file1: './sources/js/file_1.js',
  file2: './sources/js/file_2.js',
},
output: {
  path: path.resolve(__dirname, 'dist'),
  filename: './[name].js',
  library: '[name]',
  libraryExport: 'default',
  libraryTarget: 'umd', // you can use libraries everywhere, e.g requirejs, node
  umdNamedDefine: true,
},
</code>```

From now you have access to:

```<code><script>
  new file1(); // console.log show file_2
  new file2(); // console.log show file_2
</script>
</code>```

You can also now pass options to functions. Take a look at my solution

Body:
Firstly, apologies! I struggled summing up the issue in the title so please feel free to amend.

The problem

Suppose I have two ES6 files that expose default functions

// file_1.js
export default function(){ /* do the thing */ }

// file_2.js
export default function(){ /* do the other thing */ }

Now I bundle file_1 into a module using webpack (w/ babel loader) using the following output configuration

// webpack config 1.
{
   ...
   entry : '/path/to/file_1.js'
   output : {
      path: '/where/it/goes/',
      filename : 'main.js',
      library: 'my_component',
      libraryTarget: 'var'
   }
}

I also have a minimal package.json so it can be imported as a npm module { name: 'file_1', main: 'main.js' }

Now the challenge comes when I want to bundle together the code from file_1 and the module file_2 and a uniform manner.

// master_entry.js

const components = {
    file_1 : require('file_1'),
    file_2 : require('/path/to/file_2')
}

If I bundle the above and look at the resulting form of components it looks like so

console.log(components.file_1)
// outputs
Module {__esModule: true, Symbol(Symbol.toStringTag): "Module" }

console.log(components.file_2)
// outputs
Module {default: f, __esModule: true, Symbol(Symbol.toStringTag): "Module" }

Hence for file_2 (which was not prebundled) I have the default function available - which is what I want. How do I achieve the same thing when bundling file_1?

I have tried playing around with the webpack output options such as library, libraryTarget, libraryExport. However I am a bit lost and have spent far to long in the docs now - hence the cry for help!

Thanks in advance.

For clarity

file_1.js -webpack-> package file_1 (entry point file_1.js)
master_entry -webpack-> main.js

I.e, webpack is running first on file_1.js and subsequently upon the combination of importing the file_1 package and file_2.js.

Body:
I think I have a solution ;)

// file_1.js
export default function file1(){ console.log('file_1') }

// file_2.js
export default function file2(){ console.log('file_2') }

The webpack.config.js should look like this

entry: {
  file1: './sources/js/file_1.js',
  file2: './sources/js/file_2.js',
},
output: {
  path: path.resolve(__dirname, 'dist'),
  filename: './[name].js',
  library: '[name]',
  libraryExport: 'default',
  libraryTarget: 'umd', // you can use libraries everywhere, e.g requirejs, node
  umdNamedDefine: true,
},

From now you have access to:

<script>
  new file1(); // console.log show file_2
  new file2(); // console.log show file_2
</script>

You can also now pass options to functions. Take a look at my solution


问:
Syntax error with MariaDB Involving Server Version
I have been running a LiveSite website now for almost a year without any problems, but have for the last 1-2 weeks get this error:

Error: Query failed.

You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'rows, cols, informatio' at line 14

I have submitted a support ticket, but they have helped.

I have looked around my cPanel settings in phpMyAdmin at the database settings, but really don't know where to look.

I am using ( i.e. the web hosting company I use):

Database server

Server: Localhost via UNIX socket

Server type: MariaDB

Server connection: SSL is not being used
￼

Server version: 10.3.14-MariaDB-cll-lve - MariaDB Server

Protocol version: 10

User: edmbeats@localhost

Server charset: UTF-8 Unicode (utf8)

Web server

cpsrvd 11.78.0.24

Database client version: libmysql - 5.1.73

PHP extension: mysqli, curl, mbstring

PHP version: 7.2.7

phpMyAdmin

Version information: 4.8.3

Nothing really, need to know where to go to fix this. I have not changed any thing at the site for 8 months, no updates, no setting changes. I am not sure if the hosting company has updated their DB programs, and caused this. There is a newer version of LiveSite the 2019 version that is newer than  the 2017 version I am using, but the LiveSite does not support an AutoUpdate for the 2017 version within the hosting platform.

NA

答:
https://mariadb.com/kb/en/library/reserved-words/ says the keyword ROWS was added as a reserved word in MariaDB 10.2.4.

You can use reserved words in your SQL statements by delimiting them with back-ticks:

```<code>... `rows`, cols, informatio...
</code>```

So you'll have to edit your application code that runs that query, which means you'll have to figure out exactly which query caused the error.

It's a good practice to avoid using reserved words for your table and column identifiers. If the LiveSite code is using a reserved word, they should be made aware that they need to correct this, or else put back-ticks around table and column identifiers that conflict with reserved words. I suggest you contact them with a support request.

Body:
https://mariadb.com/kb/en/library/reserved-words/ says the keyword ROWS was added as a reserved word in MariaDB 10.2.4.

You can use reserved words in your SQL statements by delimiting them with back-ticks:

... `rows`, cols, informatio...

So you'll have to edit your application code that runs that query, which means you'll have to figure out exactly which query caused the error.

It's a good practice to avoid using reserved words for your table and column identifiers. If the LiveSite code is using a reserved word, they should be made aware that they need to correct this, or else put back-ticks around table and column identifiers that conflict with reserved words. I suggest you contact them with a support request.


问:
Hiding element codes when debugging on chrome?
The code I'm debugging in action is here:

https://reed123.000webhostapp.com/test/story_html5.html

If I hover my mouse over the pin of the dial I got aria-label for the pin like this:

But when I move the mouse out of the pin the aria-label of the pin hides:

The bad news is I can't select the pin with its aria-label using jQuery.

I've tried :

```<code>var item = $('[aria-label="spinner Dial 1 range from 0 to 12"]')
$(item).animate({'top': '-=30px'},'slow');
</code>```

But it seems that I can't select the pin using this aria-label!!!

Please help, I really need that aria-label to select the pin using jQuery.

Why I can't have that aria-label and how to select it?

答: I think you want to select dynamically created element. Look here - jQuery select dynamically created html element</p>

Body:
The code I'm debugging in action is here:

https://reed123.000webhostapp.com/test/story_html5.html

If I hover my mouse over the pin of the dial I got aria-label for the pin like this:

But when I move the mouse out of the pin the aria-label of the pin hides:

The bad news is I can't select the pin with its aria-label using jQuery.

I've tried :

var item = $('[aria-label="spinner Dial 1 range from 0 to 12"]')
$(item).animate({'top': '-=30px'},'slow');

But it seems that I can't select the pin using this aria-label!!!

Please help, I really need that aria-label to select the pin using jQuery.

Why I can't have that aria-label and how to select it?