问:
Parent doesn't wait for child processes to finish despite reaping
I am fully aware that there are tonnes of articles explaining the inner workings of parent-child process dynamics. I have gone through them and got my stuff working as  I want it to function, almost. But there is one thing which is bugging me out and I am not able to understand it despite multiple tries.

Problem: Despite reaping the children, main is not waiting for all children to finish and exits prematurely. I believe I did make a proper exit from the child process and I have installed the REAPER in the child process - so how is main exiting before the child finishes?

Not looking for a solution here - but I need a new direction where I could bang my head for the next week. As of now - I feel I have exhausted my options and tried a lot many things but to no avail.

Some background about what I am trying to achieve:

All in all - I want all the children to finish, and only then I want to proceed to do something further. Each child process spawns a bunch of threads and those threads are properly joined by the said child process which then proceeds to make an exit with exit(0).

The additional hoopla you might observe in the program is nothing but our requirement where we are to hit 5 APIs (engines) but only with a fixed batch size, say 10 for each, at a time. I launch child process for each engine and launch thread for each request - and then I wait for all threads to finish, join them, and only then child process exits. Only now I could deposit the next batch of requests to the same engine, and I do this for all engines till I exhaust my total requests, say 10000.

Each request may take anywhere between 1 second to 2 hours - basically they are CSV reports being fetched from an HTTP API.

My issue is that when I have exhausted my total set of requests - I am not able to wait make the MAIN wait till all child processes have finished. This is weird and is the issue which I am trying to tackle.

Any ideas?

My Program OUTPUT:

```<code>[compuser@lenovoe470:little-stuff]$  perl 07--20190526-batch-processing-using-threads-with-busy-pool-detection-2.pl 12
26710: STARTING TASKS IN BATCHES
26710: RUNNING batch_engine 1_e1 tasks (1 2)
26710: RUNNING batch_engine 2_e2 tasks (3 4)
26710: RUNNING batch_engine 3_e3 tasks (5 6 7)
26710: BUSY_ENGINE: e1.
26710: BUSY_ENGINE: e2.
26710: BUSY_ENGINE: e3.
26710: BUSY_ENGINE: e1.
26710: BUSY_ENGINE: e2.
26710:26712: TASK_ORCHESTRATOR: >> finished batch_engine (2_e2) tasks (3 4)
26710: PID (26712) has finished with status (0). updating proc hash
26710: BUSY_ENGINE: e3.
26710:26713: TASK_ORCHESTRATOR: >> finished batch_engine (3_e3) tasks (5 6 7)
26710:26711: TASK_ORCHESTRATOR: >> finished batch_engine (1_e1) tasks (1 2)
26710: PID (26713) has finished with status (0). updating proc hash
26710: BUSY_ENGINE: e1.
26710: PID (26711) has finished with status (0). updating proc hash
26710: RUNNING batch_engine 4_e2 tasks (8 9)
26710: RUNNING batch_engine 5_e3 tasks (10 11 12)
26710: FINISHED TASKS IN BATCHES
[compuser@lenovoe470:little-stuff]$  1:26722: TASK_ORCHESTRATOR: >> finished batch_engine (5_e3) tasks (10 11 12)
1:26721: TASK_ORCHESTRATOR: >> finished batch_engine (4_e2) tasks (8 9)
</code>```

In above output:

Running batch_engine means I am running a batch of numbered tasks.

BUSY_ENGINE means that the endpoint/engine is busy as it is already busy handling max batch size of requests. I need to wait.

finished batch_engine means that the child process has finished processing the given batch of requests for a particular engine/end-point. It exits and main detects that the current engine is now free and next batch can be en-queued

if we see the last 2 lines, its evident that the child processes' output has spilled over & main exited prematurely without waiting for the running children. Why? any help?

My program:

```<code>#!/usr/bin/env perl

use strict;
use warnings;
use Data::Dumper;
use POSIX ':sys_wait_h';
use Thread qw(async);

STDOUT->autoflush(1);

# doesn't work
  sub reaper {
    my $reaped;
    while (($reaped = waitpid (-1,&WNOHANG) > 0)) {
      print "$$: reaped: $reaped\n";
      sleep(1);
    }
    $SIG{CHLD} = \&reaper;
  }
# doesn't work

my @total_tasks = (1 .. shift || 9);
my @engines = (qw/e1 e2 e3/);
my $sizes = { e1 => 2, e2 => 2, e3 => 3, };

my $proc_hash;
my $global_string = "ENGINE";

# source: https://duyanghao.github.io/ways_avoid_zombie_process/
#
  sub REAPER {
    local ($!, $?);
    while ( (my $reaped_pid = waitpid(-1, WNOHANG)) > 0 ) {
      if ( WIFEXITED($?) )
      {
        # my
        my $ret_code = WEXITSTATUS($?);
        print "$$: PID ($reaped_pid) has finished with status ($ret_code). updating proc hash\n";
        my $engine_name = $proc_hash->{$reaped_pid};
        delete ($proc_hash->{$reaped_pid});
        delete ($proc_hash->{$engine_name});
        # my

        # original
        #my $ret_code = WEXITSTATUS($?);
        #print "child process:$pid exit with code:$ret_code\n";
        # original
      }
    }
  }
#

$SIG{CHLD} = \&REAPER;

sub random_sleep_time {
  return (int(rand(5)+1))
  #return (sprintf "%.2f",(rand(1)+1))
}

sub task_runner {
  my @args = @_;
  my ($batch_engine, $task) = ($args[0]->[0],$args[0]->[1]);
  STDOUT->autoflush(1);
  my $task_time = random_sleep_time();
  sleep ($task_time);
  threads->exit(0);
  #print "$$:".(threads->tid()).": TASK_RUNNER: $global_string ($batch_engine) task ($task) finished in $task_time seconds\n";
  #return;
};

sub task_orchestrator {
  my ($batch_engine, @tasks) = @_;
  my $engine = (split (/_/,$batch_engine))[1];
  my $task_orch_pid = fork();
  die "Failed to fork task_orchestrator\n" if not defined $task_orch_pid;

  if ($task_orch_pid != 0) {
    $proc_hash->{$engine} = $task_orch_pid;
    $proc_hash->{$task_orch_pid} = $engine;
  }

  if ($task_orch_pid == 0) {
    STDOUT->autoflush(1);
    my @tids;
    for (my $i=1 ; $i <= $#tasks ; $i++) { push (@tids,$i) }
    foreach my $task_number (0 .. $#tasks) {
      $tids [$task_number] = threads->create (
        \&task_runner,[$batch_engine,$tasks [$task_number]]
      );
    }
    my $ppid = getppid();
    foreach my $tid (@tids) {$tid->join()}
    print "$ppid:$$: TASK_ORCHESTRATOR: >> finished batch_engine ($batch_engine) tasks (@tasks)\n";
    exit (0);
  }
}

sub update_proc_hash {
  my $finished_pid = waitpid (-1, POSIX->WNOHANG);
  if ($finished_pid > 0) {
    print "$$: PID ($finished_pid) has finished. updating proc hash\n";
    my $engine_name = $proc_hash->{$finished_pid};
    delete ($proc_hash->{$finished_pid});
    delete ($proc_hash->{$engine_name});
  }
}

my $batch=1;
print "$$: STARTING TASKS IN BATCHES\n";
while (@total_tasks) {
  foreach my $engine (@engines) {
    update_proc_hash();
    if (exists $proc_hash->{$engine}) {
      print "$$: BUSY_ENGINE: $engine.\n";
      sleep (1);
      next;
    }
    else {
      my @engine_tasks;
      my $engine_max_tasks = $sizes->{$engine};
      while ($engine_max_tasks-- != 0) {
        my $task = shift @total_tasks;
        push (@engine_tasks,$task) if $task;
      }
      if (@engine_tasks) {
        my $batch_engine = $batch.'_'.$engine;
        print "$$: RUNNING batch_engine $batch_engine tasks (@engine_tasks)\n";
        task_orchestrator ("$batch_engine",@engine_tasks);
        $batch++;
      }
    }
  }
}

REAPER();

print "$$: FINISHED TASKS IN BATCHES\n";

__END__
</code>```

Update after 3 days: Thank you SO community. Once again, I am grateful to all of you who have taken out their time to look into this and helped spot and correct the problem. Thank you so much.

Allow me to share the new output with the final program for everyone's reference.

OUTPUT after using the fix:

```<code>User@Host:/cygdrive/c/bash-home> perl test.pl
22044: STARTING TASKS IN BATCHES
22044: MAIN: engine (e1) is RUNNING batch #1 tasks: (1 2)
22044: MAIN: engine (e2) is RUNNING batch #2 tasks: (3 4 5)
22044: MAIN: engine (e3) is RUNNING batch #3 tasks: (6 7)
41456: TASK_RUNNER: engine (e1) finished batch #1 task #1 in (1.80) seconds
41456: TASK_RUNNER: engine (e1) finished batch #1 task #2 in (1.31) seconds
41456: TASK_ORCHESTRATOR: engine (e1) finished batch #1 tasks in (1.00) seconds.
22044: REAPER: TASK_ORCHESTRATOR pid (41456) has finished with status (0).
18252: TASK_RUNNER: engine (e2) finished batch #2 task #3 in (1.04) seconds
18252: TASK_RUNNER: engine (e2) finished batch #2 task #4 in (1.91) seconds
18252: TASK_RUNNER: engine (e2) finished batch #2 task #5 in (1.63) seconds
18252: TASK_ORCHESTRATOR: engine (e2) finished batch #2 tasks in (1.00) seconds.
22044: REAPER: TASK_ORCHESTRATOR pid (18252) has finished with status (0).
14544: TASK_RUNNER: engine (e3) finished batch #3 task #6 in (1.42) seconds
14544: TASK_RUNNER: engine (e3) finished batch #3 task #7 in (1.84) seconds
14544: TASK_ORCHESTRATOR: engine (e3) finished batch #3 tasks in (1.00) seconds.
22044: REAPER: TASK_ORCHESTRATOR pid (14544) has finished with status (0).
22044: MAIN: engine (e1) is RUNNING batch #4 tasks: (8 9)
22044: MAIN: engine (e2) is RUNNING batch #5 tasks: (10)
37612: TASK_RUNNER: engine (e1) finished batch #4 task #8 in (1.19) seconds
37612: TASK_RUNNER: engine (e1) finished batch #4 task #9 in (1.31) seconds
37612: TASK_ORCHESTRATOR: engine (e1) finished batch #4 tasks in (1.00) seconds.
16300: TASK_RUNNER: engine (e2) finished batch #5 task #10 in (1.53) seconds
16300: TASK_ORCHESTRATOR: engine (e2) finished batch #5 tasks in (1.00) seconds.
22044: ALL ORCHESTRATORS HAVE FINISHED
22044: FINISHED TASKS IN BATCHES
</code>```

FINAL Working PROGRAM:

```<code>#!/usr/bin/env perl

use strict;
use warnings;
use Data::Dumper;
use POSIX ':sys_wait_h';
use threads;

STDOUT->autoflush(1);

my @total_tasks = (1 .. 10);
my $sleep_time = 1;
my @engines = (qw/e1 e2 e3/);
my $sizes = {
  e1 => 2,
  e2 => 3,
  e3 => 2,
};

my $proc_hash;
my $global_string = "engine";

sub REAPER {
  local ($!, $?);
  while ( (my $reaped_pid = waitpid(-1, WNOHANG)) > 0 ) {
    if ( WIFEXITED($?) ) {
      my $ret_code = WEXITSTATUS($?);
      print "$$: REAPER: TASK_ORCHESTRATOR pid ($reaped_pid) has finished with status ($ret_code).\n";
      my $engine_name = $proc_hash->{$reaped_pid};
      delete ($proc_hash->{$reaped_pid});
      delete ($proc_hash->{$engine_name});
    }
  }
}

$SIG{CHLD} = \&REAPER;

sub random_sleep_time { return sprintf ("%.2f",(rand ($sleep_time||5) + 1)) }

sub task_runner {
  STDOUT->autoflush(1);
  my @args = @_;
  my ($batch_engine, $task) = ($args[0]->[0],$args[0]->[1]);
  my ($batch, $engine) = split (/_/,$batch_engine);
  my $task_time = random_sleep_time();
  sleep ($task_time);
  print "$$: TASK_RUNNER: $global_string ($engine) finished batch #$batch task #$task in ($task_time) seconds\n";
  threads->exit(0);
};

sub task_orchestrator {
  my ($batch_engine, @tasks) = @_;
  my ($batch, $engine) = split (/_/,$batch_engine);
  my $task_orch_pid = fork();
  die "Failed to fork task_orchestrator\n" if not defined $task_orch_pid;

  if ($task_orch_pid != 0) {
    $proc_hash->{$engine} = $task_orch_pid;
    $proc_hash->{$task_orch_pid} = $engine;
  }

  if ($task_orch_pid == 0) {
    STDOUT->autoflush(1);
    my @tids;
    my $start_time = time;
    for (my $i=1 ; $i <= $#tasks ; $i++) { push (@tids,$i) }
    foreach my $task_number (0 .. $#tasks) {
      $tids [$task_number] = threads->create (
        \&task_runner,[$batch_engine,$tasks [$task_number]]
      );
    }
    foreach my $tid (@tids) {$tid->join()}
    my $end_time = time;
    my $total_time = sprintf ("%.2f",($end_time - $start_time));
    print "$$: TASK_ORCHESTRATOR: engine ($engine) finished batch #$batch tasks in ($total_time) seconds.\n";
    exit (0);
  }
}

my $batch=1;
print "$$: STARTING TASKS IN BATCHES\n";
while (@total_tasks)
{
  foreach my $engine (@engines)
  {
    if (exists $proc_hash->{$engine})
    {
      sleep (1);
      next;
    }
    else
    {
      my @engine_tasks;
      my $engine_max_tasks = $sizes->{$engine};
      while ($engine_max_tasks-- != 0)
      {
        my $task = shift @total_tasks;
        push (@engine_tasks,$task) if $task;
      }
      if (@engine_tasks)
      {
        my $batch_engine = $batch.'_'.$engine;
        print "$$: MAIN: engine ($engine) is RUNNING batch #$batch tasks: (@engine_tasks)\n";
        task_orchestrator ($batch_engine,@engine_tasks);
        $batch++;
      }
    }
  }
}

# All 3 below work properly
#sleep (.2) while ((waitpid(-1, WNOHANG)) >= 0);
#sleep (.2) while ((waitpid(-1, WNOHANG)) != -1);
sleep (.2) while ((waitpid(-1, WNOHANG)) > -1);

print "$$: ALL ORCHESTRATORS HAVE FINISHED\n";
print "$$: FINISHED TASKS IN BATCHES\n";
__END__
</code>```

答:
On exiting your main loop, you call REAPER(), which does a non-blocking waitpid(). Non-blocking. Non. And it's not blocking. So it's exiting.

While I'm here, I note that your update_proc_hash() function is not looping like the other things that do waitpid(), so it's not catching all that it could. Do yourself a favour and factor all that stuff out neatly.

Body:
I am fully aware that there are tonnes of articles explaining the inner workings of parent-child process dynamics. I have gone through them and got my stuff working as  I want it to function, almost. But there is one thing which is bugging me out and I am not able to understand it despite multiple tries.

Problem: Despite reaping the children, main is not waiting for all children to finish and exits prematurely. I believe I did make a proper exit from the child process and I have installed the REAPER in the child process - so how is main exiting before the child finishes?

Not looking for a solution here - but I need a new direction where I could bang my head for the next week. As of now - I feel I have exhausted my options and tried a lot many things but to no avail.

Some background about what I am trying to achieve:

All in all - I want all the children to finish, and only then I want to proceed to do something further. Each child process spawns a bunch of threads and those threads are properly joined by the said child process which then proceeds to make an exit with exit(0).

The additional hoopla you might observe in the program is nothing but our requirement where we are to hit 5 APIs (engines) but only with a fixed batch size, say 10 for each, at a time. I launch child process for each engine and launch thread for each request - and then I wait for all threads to finish, join them, and only then child process exits. Only now I could deposit the next batch of requests to the same engine, and I do this for all engines till I exhaust my total requests, say 10000.

Each request may take anywhere between 1 second to 2 hours - basically they are CSV reports being fetched from an HTTP API.

My issue is that when I have exhausted my total set of requests - I am not able to wait make the MAIN wait till all child processes have finished. This is weird and is the issue which I am trying to tackle.

Any ideas?

My Program OUTPUT:

[compuser@lenovoe470:little-stuff]$  perl 07--20190526-batch-processing-using-threads-with-busy-pool-detection-2.pl 12
26710: STARTING TASKS IN BATCHES
26710: RUNNING batch_engine 1_e1 tasks (1 2)
26710: RUNNING batch_engine 2_e2 tasks (3 4)
26710: RUNNING batch_engine 3_e3 tasks (5 6 7)
26710: BUSY_ENGINE: e1.
26710: BUSY_ENGINE: e2.
26710: BUSY_ENGINE: e3.
26710: BUSY_ENGINE: e1.
26710: BUSY_ENGINE: e2.
26710:26712: TASK_ORCHESTRATOR: >> finished batch_engine (2_e2) tasks (3 4)
26710: PID (26712) has finished with status (0). updating proc hash
26710: BUSY_ENGINE: e3.
26710:26713: TASK_ORCHESTRATOR: >> finished batch_engine (3_e3) tasks (5 6 7)
26710:26711: TASK_ORCHESTRATOR: >> finished batch_engine (1_e1) tasks (1 2)
26710: PID (26713) has finished with status (0). updating proc hash
26710: BUSY_ENGINE: e1.
26710: PID (26711) has finished with status (0). updating proc hash
26710: RUNNING batch_engine 4_e2 tasks (8 9)
26710: RUNNING batch_engine 5_e3 tasks (10 11 12)
26710: FINISHED TASKS IN BATCHES
[compuser@lenovoe470:little-stuff]$  1:26722: TASK_ORCHESTRATOR: >> finished batch_engine (5_e3) tasks (10 11 12)
1:26721: TASK_ORCHESTRATOR: >> finished batch_engine (4_e2) tasks (8 9)

In above output:

Running batch_engine means I am running a batch of numbered tasks.

BUSY_ENGINE means that the endpoint/engine is busy as it is already busy handling max batch size of requests. I need to wait.

finished batch_engine means that the child process has finished processing the given batch of requests for a particular engine/end-point. It exits and main detects that the current engine is now free and next batch can be en-queued

if we see the last 2 lines, its evident that the child processes' output has spilled over & main exited prematurely without waiting for the running children. Why? any help?

My program:

#!/usr/bin/env perl

use strict;
use warnings;
use Data::Dumper;
use POSIX ':sys_wait_h';
use Thread qw(async);

STDOUT->autoflush(1);

# doesn't work
  sub reaper {
    my $reaped;
    while (($reaped = waitpid (-1,&WNOHANG) > 0)) {
      print "$$: reaped: $reaped\n";
      sleep(1);
    }
    $SIG{CHLD} = \&reaper;
  }
# doesn't work

my @total_tasks = (1 .. shift || 9);
my @engines = (qw/e1 e2 e3/);
my $sizes = { e1 => 2, e2 => 2, e3 => 3, };

my $proc_hash;
my $global_string = "ENGINE";

# source: https://duyanghao.github.io/ways_avoid_zombie_process/
#
  sub REAPER {
    local ($!, $?);
    while ( (my $reaped_pid = waitpid(-1, WNOHANG)) > 0 ) {
      if ( WIFEXITED($?) )
      {
        # my
        my $ret_code = WEXITSTATUS($?);
        print "$$: PID ($reaped_pid) has finished with status ($ret_code). updating proc hash\n";
        my $engine_name = $proc_hash->{$reaped_pid};
        delete ($proc_hash->{$reaped_pid});
        delete ($proc_hash->{$engine_name});
        # my

        # original
        #my $ret_code = WEXITSTATUS($?);
        #print "child process:$pid exit with code:$ret_code\n";
        # original
      }
    }
  }
#

$SIG{CHLD} = \&REAPER;

sub random_sleep_time {
  return (int(rand(5)+1))
  #return (sprintf "%.2f",(rand(1)+1))
}

sub task_runner {
  my @args = @_;
  my ($batch_engine, $task) = ($args[0]->[0],$args[0]->[1]);
  STDOUT->autoflush(1);
  my $task_time = random_sleep_time();
  sleep ($task_time);
  threads->exit(0);
  #print "$$:".(threads->tid()).": TASK_RUNNER: $global_string ($batch_engine) task ($task) finished in $task_time seconds\n";
  #return;
};

sub task_orchestrator {
  my ($batch_engine, @tasks) = @_;
  my $engine = (split (/_/,$batch_engine))[1];
  my $task_orch_pid = fork();
  die "Failed to fork task_orchestrator\n" if not defined $task_orch_pid;

  if ($task_orch_pid != 0) {
    $proc_hash->{$engine} = $task_orch_pid;
    $proc_hash->{$task_orch_pid} = $engine;
  }

  if ($task_orch_pid == 0) {
    STDOUT->autoflush(1);
    my @tids;
    for (my $i=1 ; $i <= $#tasks ; $i++) { push (@tids,$i) }
    foreach my $task_number (0 .. $#tasks) {
      $tids [$task_number] = threads->create (
        \&task_runner,[$batch_engine,$tasks [$task_number]]
      );
    }
    my $ppid = getppid();
    foreach my $tid (@tids) {$tid->join()}
    print "$ppid:$$: TASK_ORCHESTRATOR: >> finished batch_engine ($batch_engine) tasks (@tasks)\n";
    exit (0);
  }
}

sub update_proc_hash {
  my $finished_pid = waitpid (-1, POSIX->WNOHANG);
  if ($finished_pid > 0) {
    print "$$: PID ($finished_pid) has finished. updating proc hash\n";
    my $engine_name = $proc_hash->{$finished_pid};
    delete ($proc_hash->{$finished_pid});
    delete ($proc_hash->{$engine_name});
  }
}

my $batch=1;
print "$$: STARTING TASKS IN BATCHES\n";
while (@total_tasks) {
  foreach my $engine (@engines) {
    update_proc_hash();
    if (exists $proc_hash->{$engine}) {
      print "$$: BUSY_ENGINE: $engine.\n";
      sleep (1);
      next;
    }
    else {
      my @engine_tasks;
      my $engine_max_tasks = $sizes->{$engine};
      while ($engine_max_tasks-- != 0) {
        my $task = shift @total_tasks;
        push (@engine_tasks,$task) if $task;
      }
      if (@engine_tasks) {
        my $batch_engine = $batch.'_'.$engine;
        print "$$: RUNNING batch_engine $batch_engine tasks (@engine_tasks)\n";
        task_orchestrator ("$batch_engine",@engine_tasks);
        $batch++;
      }
    }
  }
}

REAPER();

print "$$: FINISHED TASKS IN BATCHES\n";

__END__

Update after 3 days: Thank you SO community. Once again, I am grateful to all of you who have taken out their time to look into this and helped spot and correct the problem. Thank you so much.

Allow me to share the new output with the final program for everyone's reference.

OUTPUT after using the fix:

User@Host:/cygdrive/c/bash-home> perl test.pl
22044: STARTING TASKS IN BATCHES
22044: MAIN: engine (e1) is RUNNING batch #1 tasks: (1 2)
22044: MAIN: engine (e2) is RUNNING batch #2 tasks: (3 4 5)
22044: MAIN: engine (e3) is RUNNING batch #3 tasks: (6 7)
41456: TASK_RUNNER: engine (e1) finished batch #1 task #1 in (1.80) seconds
41456: TASK_RUNNER: engine (e1) finished batch #1 task #2 in (1.31) seconds
41456: TASK_ORCHESTRATOR: engine (e1) finished batch #1 tasks in (1.00) seconds.
22044: REAPER: TASK_ORCHESTRATOR pid (41456) has finished with status (0).
18252: TASK_RUNNER: engine (e2) finished batch #2 task #3 in (1.04) seconds
18252: TASK_RUNNER: engine (e2) finished batch #2 task #4 in (1.91) seconds
18252: TASK_RUNNER: engine (e2) finished batch #2 task #5 in (1.63) seconds
18252: TASK_ORCHESTRATOR: engine (e2) finished batch #2 tasks in (1.00) seconds.
22044: REAPER: TASK_ORCHESTRATOR pid (18252) has finished with status (0).
14544: TASK_RUNNER: engine (e3) finished batch #3 task #6 in (1.42) seconds
14544: TASK_RUNNER: engine (e3) finished batch #3 task #7 in (1.84) seconds
14544: TASK_ORCHESTRATOR: engine (e3) finished batch #3 tasks in (1.00) seconds.
22044: REAPER: TASK_ORCHESTRATOR pid (14544) has finished with status (0).
22044: MAIN: engine (e1) is RUNNING batch #4 tasks: (8 9)
22044: MAIN: engine (e2) is RUNNING batch #5 tasks: (10)
37612: TASK_RUNNER: engine (e1) finished batch #4 task #8 in (1.19) seconds
37612: TASK_RUNNER: engine (e1) finished batch #4 task #9 in (1.31) seconds
37612: TASK_ORCHESTRATOR: engine (e1) finished batch #4 tasks in (1.00) seconds.
16300: TASK_RUNNER: engine (e2) finished batch #5 task #10 in (1.53) seconds
16300: TASK_ORCHESTRATOR: engine (e2) finished batch #5 tasks in (1.00) seconds.
22044: ALL ORCHESTRATORS HAVE FINISHED
22044: FINISHED TASKS IN BATCHES

FINAL Working PROGRAM:

#!/usr/bin/env perl

use strict;
use warnings;
use Data::Dumper;
use POSIX ':sys_wait_h';
use threads;

STDOUT->autoflush(1);

my @total_tasks = (1 .. 10);
my $sleep_time = 1;
my @engines = (qw/e1 e2 e3/);
my $sizes = {
  e1 => 2,
  e2 => 3,
  e3 => 2,
};

my $proc_hash;
my $global_string = "engine";

sub REAPER {
  local ($!, $?);
  while ( (my $reaped_pid = waitpid(-1, WNOHANG)) > 0 ) {
    if ( WIFEXITED($?) ) {
      my $ret_code = WEXITSTATUS($?);
      print "$$: REAPER: TASK_ORCHESTRATOR pid ($reaped_pid) has finished with status ($ret_code).\n";
      my $engine_name = $proc_hash->{$reaped_pid};
      delete ($proc_hash->{$reaped_pid});
      delete ($proc_hash->{$engine_name});
    }
  }
}

$SIG{CHLD} = \&REAPER;

sub random_sleep_time { return sprintf ("%.2f",(rand ($sleep_time||5) + 1)) }

sub task_runner {
  STDOUT->autoflush(1);
  my @args = @_;
  my ($batch_engine, $task) = ($args[0]->[0],$args[0]->[1]);
  my ($batch, $engine) = split (/_/,$batch_engine);
  my $task_time = random_sleep_time();
  sleep ($task_time);
  print "$$: TASK_RUNNER: $global_string ($engine) finished batch #$batch task #$task in ($task_time) seconds\n";
  threads->exit(0);
};

sub task_orchestrator {
  my ($batch_engine, @tasks) = @_;
  my ($batch, $engine) = split (/_/,$batch_engine);
  my $task_orch_pid = fork();
  die "Failed to fork task_orchestrator\n" if not defined $task_orch_pid;

  if ($task_orch_pid != 0) {
    $proc_hash->{$engine} = $task_orch_pid;
    $proc_hash->{$task_orch_pid} = $engine;
  }

  if ($task_orch_pid == 0) {
    STDOUT->autoflush(1);
    my @tids;
    my $start_time = time;
    for (my $i=1 ; $i <= $#tasks ; $i++) { push (@tids,$i) }
    foreach my $task_number (0 .. $#tasks) {
      $tids [$task_number] = threads->create (
        \&task_runner,[$batch_engine,$tasks [$task_number]]
      );
    }
    foreach my $tid (@tids) {$tid->join()}
    my $end_time = time;
    my $total_time = sprintf ("%.2f",($end_time - $start_time));
    print "$$: TASK_ORCHESTRATOR: engine ($engine) finished batch #$batch tasks in ($total_time) seconds.\n";
    exit (0);
  }
}

my $batch=1;
print "$$: STARTING TASKS IN BATCHES\n";
while (@total_tasks)
{
  foreach my $engine (@engines)
  {
    if (exists $proc_hash->{$engine})
    {
      sleep (1);
      next;
    }
    else
    {
      my @engine_tasks;
      my $engine_max_tasks = $sizes->{$engine};
      while ($engine_max_tasks-- != 0)
      {
        my $task = shift @total_tasks;
        push (@engine_tasks,$task) if $task;
      }
      if (@engine_tasks)
      {
        my $batch_engine = $batch.'_'.$engine;
        print "$$: MAIN: engine ($engine) is RUNNING batch #$batch tasks: (@engine_tasks)\n";
        task_orchestrator ($batch_engine,@engine_tasks);
        $batch++;
      }
    }
  }
}

# All 3 below work properly
#sleep (.2) while ((waitpid(-1, WNOHANG)) >= 0);
#sleep (.2) while ((waitpid(-1, WNOHANG)) != -1);
sleep (.2) while ((waitpid(-1, WNOHANG)) > -1);

print "$$: ALL ORCHESTRATORS HAVE FINISHED\n";
print "$$: FINISHED TASKS IN BATCHES\n";
__END__


来源: stackexchange

问:
Can't seem to be able to force y axis to not be in exponent notation despite specifying it that way in my script. X axis works fine
I'm making a scatterplot and want my x and y axes to not be displayed in scientific notation. When I put in the line of code which should fix this, only the x-axis is fixed, the y-axis is left untouched.

Tried using ax.XAxis.Exponent = 0;. Works for X but ax.YAxis.Exponent = 0; doesn't work. Also came across XRuler instead, but that didn't work either. Maybe it's worth noting that changing the y-axis exponent to anything other than zero works perfectly as it should.

%-------------Make Plots-------------%

figure;
scatter(x, y, 30, magComp);
title('Compensated');

%---------------Plot Options----------------%

% axis equal
ax = gca;
ax.XAxis.Exponent = 0;
ax.YAxis.Exponent = 0;
c = colorbar();
c.Label.String = 'nT'
xlabel('Easting');
ylabel('Northing');
caxis([5.9e4 6e4]);
```

Any insight? Am new to MATLAB, so sorry for crappy code formatting. Here is an example of the output I'm getting (data boxed because it is IP). The x-axis is right, my y-axis is still in scientific notation.

答:
I think I found a solution.

Add the following line to your code:

```<code>ax.YAxis.TickLabelFormat = '%d';
</code>```

[I don't know why ax.YAxis.Exponent = 0; doesn't work]

ContentLicense: CC BY-SA 4.0

Body:
I'm making a scatterplot and want my x and y axes to not be displayed in scientific notation. When I put in the line of code which should fix this, only the x-axis is fixed, the y-axis is left untouched.

Tried using ax.XAxis.Exponent = 0;. Works for X but ax.YAxis.Exponent = 0; doesn't work. Also came across XRuler instead, but that didn't work either. Maybe it's worth noting that changing the y-axis exponent to anything other than zero works perfectly as it should.

%-------------Make Plots-------------%

figure;
scatter(x, y, 30, magComp);
title('Compensated');

%---------------Plot Options----------------%

% axis equal
ax = gca;
ax.XAxis.Exponent = 0;
ax.YAxis.Exponent = 0;
c = colorbar();
c.Label.String = 'nT'
xlabel('Easting');
ylabel('Northing');
caxis([5.9e4 6e4]);

Any insight? Am new to MATLAB, so sorry for crappy code formatting. Here is an example of the output I'm getting (data boxed because it is IP). The x-axis is right, my y-axis is still in scientific notation.

Body:
I think I found a solution.

Add the following line to your code:

ax.YAxis.TickLabelFormat = '%d';

[I don't know why ax.YAxis.Exponent = 0; doesn't work]


问:
Can multiple service perform work parallelly at the same time
I am developing an app which has multiple features which need to perform work in the background.

Each feature needs to run at the same time and perform some work parallelly.

Currently, I am using JobIntentService to perform work but I have noticed that when I try to run multiple JobIntentServices then only one run while others are made to wait and executed one by one.

I am planning to use the Service. I want to know can 5 services run at the same time and perform work parallelly?

If no then is there any alternative?

Any help would be grateful.

答: Yes they should be able to run parallel (as pointed out by @Omkar), depending on the work load that the device is already having. The reason your JobIntentServices weren’t running parallel is because the OS will choose when to call those services when it thinks is best, thus sometimes they won’t run “right away”, using a Service would be the best solution for parallel running work that needs to be ran ASAP</p>


问:
how to remember inputs value by using local storage?
I have tried to use local storage to remember the input's value after refreshing page. but my code does not work.

here is the HTML code

```<code><input type="text" name="name" onkeyup="saveValue(event)"/>
<input type="text" name="name" onkeyup="saveValue(event)"/>
<input type="text" name="age" onkeyup="saveValue(event)"/>
</code>```

and here is javascript

```<code><script type="text/javascript">
var nameArr = ["name"];
var inputs = document.getElementsByName('name');
inputs.forEach(function(el){
    el.value = getSavedValue(el);
})

function saveValue(e) {
  var name = e.target.name;
  var val = e.target.value;
  localStorage.setItem(name, val);
}

function getSavedValue(v) {
  if (!localStorage.getItem(v)) {
    return "";
  }
  return localStorage.getItem(v);
}
</script>
</code>```

if there is a way to solve this problem please tell me.
and if there is a way to do that with jquery I will be thankful to tell me that.

答:
On your code you are passing the input object as a parameter instead of its name (or value; you choose). As localStorage only stores String key-value pairs, it won't work as you're trying to find a key that is an object.

in the forEach instead of:

el.value = getSavedValue(el);

set:

el.value = getSavedValue(el.name);

or let the "getSavedValue" function accept an object as parameter, but to access localStorage you must pass a string as the key.

Body:
I have tried to use local storage to remember the input's value after refreshing page. but my code does not work.

here is the HTML code

<input type="text" name="name" onkeyup="saveValue(event)"/>
<input type="text" name="name" onkeyup="saveValue(event)"/>
<input type="text" name="age" onkeyup="saveValue(event)"/>

and here is javascript

if there is a way to solve this problem please tell me.
and if there is a way to do that with jquery I will be thankful to tell me that.


问:
Enum to dictionary and format each name
I need  to convert enum to dictionary and after that format each name (value) of dictionary.

```<code>public static Dictionary<int, string> EnumToDictionary<TK>(Func<TK, string > func)
      {
        if (typeof(TK).BaseType != typeof(Enum))
              throw new InvalidCastException();

           return Enum.GetValues(
                typeof(TK))
                .Cast<Int32>()
                .ToDictionary(
                        currentItem => currentItem => Enum.GetName(typeof(TK), currentItem))
                 /*. func for each name */;
           }

      public enum Types {

            type1 = 0,
            type2 = 1,
            type3 = 2
         }

       public string FormatName(Types t) {
             switch (t) {
                         case Types.type1:
                                return "mytype1";

                         case Types.type2:
                              return "mytype2";

                         case Types.type3:
                              return "mytype3";

                           default:
                                return string.Empty;
                 }
           }
</code>```

After that I need to do something like this:

```<code>var resultedAndFormatedDictionary =
EnumToDictionary<Types>(/*delegate FormatName() for each element of dictionary() */);
</code>```

How do I define and implement delegate (Func<object, string > func), which performs some action  for each value of dictionary?

UPDATE:
Correspoding result is

```<code>var a = EnumToDictionary<Types>(FormatName);

   //a[0] == "mytype1"
   //a[1] == "mytype2"
   //a[2] == "mytype3"
</code>```

答:
Guessing from your question you want to achieve to create a dictionary from an enum where the enum value as int is the key and some formatted name for it is the value, right?

Well, first the function you pass in should take TK as argument, shouldn't it?

Second, throwing a InvalidCastException seems a bit strange. An InvalidOperationException might be more appropriate (see also this question)

Third ToDictionary is already quite close:

```<code>public static Dictionary<int, string> EnumToDictionary<TK>(Func<TK, string> func)
{
     if (typeof(TK).BaseType != typeof(Enum))
        throw new InvalidOperationException("Type must be enum");

     return Enum.GetValues(typeof(TK)).Cast<TK>().ToDictionary(x => Convert.ToInt32(x), x => func(x));
}
</code>```

Now you can call it like this:

```<code>public enum Types
{
    type1 = 0,
    type2 = 1,
    type3 = 2
}

public string FormatName(Types t)
{
    switch (t)
    {
        case Types.type1:
            return "mytype1";

        case Types.type2:
            return "mytype2";

        case Types.type3:
             return "mytype3";

        default:
            return string.Empty;
     }
}

var resultedAndFormatedDictionary = EnumToDictionary<Types>(FormatName);
</code>```

ContentLicense: CC BY-SA 3.0

Title: Enum to dictionary and format each name

Body:
I need  to convert enum to dictionary and after that format each name (value) of dictionary.

public static Dictionary<int, string> EnumToDictionary<TK>(Func<TK, string > func)
      {
        if (typeof(TK).BaseType != typeof(Enum))
              throw new InvalidCastException();

           return Enum.GetValues(
                typeof(TK))
                .Cast<Int32>()
                .ToDictionary(
                        currentItem => currentItem => Enum.GetName(typeof(TK), currentItem))
                 /*. func for each name */;
           }

      public enum Types {

            type1 = 0,
            type2 = 1,
            type3 = 2
         }

       public string FormatName(Types t) {
             switch (t) {
                         case Types.type1:
                                return "mytype1";

                         case Types.type2:
                              return "mytype2";

                         case Types.type3:
                              return "mytype3";

                           default:
                                return string.Empty;
                 }
           }

After that I need to do something like this:

var resultedAndFormatedDictionary =
EnumToDictionary<Types>(/*delegate FormatName() for each element of dictionary() */);

How do I define and implement delegate (Func<object, string > func), which performs some action  for each value of dictionary?

UPDATE:
Correspoding result is

var a = EnumToDictionary<Types>(FormatName);

   //a[0] == "mytype1"
   //a[1] == "mytype2"
   //a[2] == "mytype3"

Body:
Guessing from your question you want to achieve to create a dictionary from an enum where the enum value as int is the key and some formatted name for it is the value, right?

Well, first the function you pass in should take TK as argument, shouldn't it?

Second, throwing a InvalidCastException seems a bit strange. An InvalidOperationException might be more appropriate (see also this question)

Third ToDictionary is already quite close:

public static Dictionary<int, string> EnumToDictionary<TK>(Func<TK, string> func)
{
     if (typeof(TK).BaseType != typeof(Enum))
        throw new InvalidOperationException("Type must be enum");

     return Enum.GetValues(typeof(TK)).Cast<TK>().ToDictionary(x => Convert.ToInt32(x), x => func(x));
}

Now you can call it like this:

public enum Types
{
    type1 = 0,
    type2 = 1,
    type3 = 2
}

public string FormatName(Types t)
{
    switch (t)
    {
        case Types.type1:
            return "mytype1";

        case Types.type2:
            return "mytype2";

        case Types.type3:
             return "mytype3";

        default:
            return string.Empty;
     }
}

var resultedAndFormatedDictionary = EnumToDictionary<Types>(FormatName);


问:
Row-wise concatenation of hundreds of csv files into single dataframe
I have hundreds of csv files - each corresponding to a unique chemical. All the csv files have the same format (of 3 columns and values within the columns for each chemical).

I would like to combine all these files via a row-wise concatenation into a single pandas dataframe but not have the header columns from each csv file in the final dataframe.  I am using the following Python code but getting an error from the pd.read_csv section -

"EmptyDataError: No columns to parse from file"

Code follows below.

```<code>files_path=r"C:\Users\Desktop\Python\RWE_350files_merge\Drugs"
csvfiles=glob.glob(os.path.join(files_path, "*.csv"))

master_df = []
for file in csvfiles:
    df = pd.read_csv(file, header = 0)
    master_df.append(df)

result = pd.concat(master_df, ignore_index=True)
</code>```

I know the csv files are not "empty" as I can concatenate them from the command line successfully - however, this method retains the header columns from each csv file into the final "concatenated" dataframe - so this is not really acceptable.

How can I fix this issue?

答:
This looks fine:

```<code>    df = pd.read_csv(file, header=0)
</code>```

But apparently some of your input files are empty.
Adding in a print(file) debug statement
would help you to focus on particular ones that are empty.

You could Look Before You Leap:

```<code>    threshold = 2
    if os.path.getsize(file) > threshold:
        df = pd.read_csv(file, header=0)
        master_df.append(df)
</code>```

Or you could decide that it is Easier To Ask Forgiveness Than Permission:

```<code>    try:
        df = pd.read...
    except pandas.errors.EmptyDataError:
        print(file, 'was empty. Continuing...')
</code>```

Body:
I have hundreds of csv files - each corresponding to a unique chemical. All the csv files have the same format (of 3 columns and values within the columns for each chemical).

I would like to combine all these files via a row-wise concatenation into a single pandas dataframe but not have the header columns from each csv file in the final dataframe.  I am using the following Python code but getting an error from the pd.read_csv section -

"EmptyDataError: No columns to parse from file"

Code follows below.

files_path=r"C:\Users\Desktop\Python\RWE_350files_merge\Drugs"
csvfiles=glob.glob(os.path.join(files_path, "*.csv"))

master_df = []
for file in csvfiles:
    df = pd.read_csv(file, header = 0)
    master_df.append(df)

result = pd.concat(master_df, ignore_index=True)

I know the csv files are not "empty" as I can concatenate them from the command line successfully - however, this method retains the header columns from each csv file into the final "concatenated" dataframe - so this is not really acceptable.

How can I fix this issue?

Body:
This looks fine:

    df = pd.read_csv(file, header=0)

But apparently some of your input files are empty.
Adding in a print(file) debug statement
would help you to focus on particular ones that are empty.

You could Look Before You Leap:

    threshold = 2
    if os.path.getsize(file) > threshold:
        df = pd.read_csv(file, header=0)
        master_df.append(df)

Or you could decide that it is Easier To Ask Forgiveness Than Permission:

    try:
        df = pd.read...
    except pandas.errors.EmptyDataError:
        print(file, 'was empty. Continuing...')


问:
Convert cURL command to python using requests
I am using deepspeech and deespeech-server. I am able to send the cURL command:

curl -X POST --data-binary @what_time_is_it.wav http://localhost:8080/stt
```

<p>This gives me the correct speech to text translation "what time is it". </p>

<p>I am now trying to achieve the same result with a python script. My code is:</p>

<pre class="lang-py prettyprint-override"><code>import requests

data = {'data-binary': '@what_time_is_it.wav'}
response = requests.post("http://localhost:8080/stt", data=data)
print(response.content)
print(response.text)
</code>```

I get the following output:

b'Speech to text error'
```

<p>And on my server I get: </p>

<pre class="lang-none prettyprint-override"><code>STT error: File format b'data'... not understood.
</code>```

Does anyone have ideas for how I might fix this?

答:
Try something like this:

import requests

filepath = '/path/to/what_time_is_it.wav'
r = requests.post("http://localhost:8080/stt", data=open(filepath).read())
print(r.status_code)
print(r.content)
print(r.text)
```

Body:
I am using deepspeech and deespeech-server. I am able to send the cURL command:

curl -X POST --data-binary @what_time_is_it.wav http://localhost:8080/stt

This gives me the correct speech to text translation "what time is it".

I am now trying to achieve the same result with a python script. My code is:

import requests

data = {'data-binary': '@what_time_is_it.wav'}
response = requests.post("http://localhost:8080/stt", data=data)
print(response.content)
print(response.text)

I get the following output:

b'Speech to text error'

And on my server I get:

STT error: File format b'data'... not understood.

Does anyone have ideas for how I might fix this?


问:
Why do I get duplicate tuples in my query?
I am building a library database.

I want to write a query that returns the available books on the library(their isbns,titles,author names,pubname,categoryName and the Number of copies available.

My query involves the following relations (with bold are the primary keys) :

Book  (ISBN, title,pubYear,numpages, pubName ) pubName FK to publisher

copies  (ISBN, copyNr, shelf ) ISBN FK to Book

borrows (memberID, ISBN, copyNr, date_of_borrowing, date_of_return ) memberID FK to member, ISBN FK to Books, (ISBN, copyNr) FK to copies

belongs_to  (ISBN, categoryName ) SBN FK to Book, categoryName FK to category

written_by  (ISBN, authID ) ISBN FK to Book, authID FK to author

author  (authID, AFirst, ALast, Abirthdate )

My approach is the following:

```<code>SELECT b.isbn, b.title,a.ALast, a.ALast, b.pubName, be.categoryName , COUNT(b.isbn) as Number_of_copies_available
FROM copies as c
INNER JOIN book as b ON c.isbn = b.isbn
INNER JOIN belong_to as be ON b.isbn = be.isbn
INNER JOIN written_by as w ON w.isbn=b.isbn
INNER JOIN author as a ON a.authID = w.authID
WHERE (c.isbn,c.copyNr) NOT IN (SELECT isbn, copyNr FROM borrows)
GROUP BY b.isbn
ORDER BY be.categoryName
</code>```

The problem is that I get duplicate tuples on the result before grouping which leads to more items per group.

For instance for some books I get the attribute value of Number_of_copies_available double than expected.

If I don't JOIN with the relations "written_by" and "author" then the result is correct. However, I also want to appear the author names in the result. What's my mistake? Thanks in advance!

答:
Don't join the tables directly when they are not perfectly related. A book's category is not really related to a book's author. With two authors and two categories, which author would you combine with which category? It makes no sense to combine them (and possibly end up with all combinations). So, aggregate first and then join the aggregates. Here is an example:</p>

```<code>select
  b.isbn,
  b.title,
  b.pubName,
  aut.authors,
  cat.categories,
  cop.total - coalesce(bor.total, 0) as available
from book b
join
(
  select w.isbn, group_concat(a.alast) as authors
  from  written_by w
  join author a ON a.authID = w.authID
  group by w.isbn
) aut ON aut.isbn = b.isbn
join
(
  select isbn, group_concat(categoryname) as categories
  from  belong_to
  group by isbn
) cat ON cat.isbn = b.isbn
join
(
  select isbn, count(*) as total
  from  copies
  group by isbn
) cop ON cop.isbn = b.isbn
left join
(
  select isbn, count(*) as total
  from  borrows
  where date_of_return > current_date
  group by isbn
) bor ON bor.isbn = b.isbn
order by b.isbn;
</code>```

Body:
I am building a library database.

I want to write a query that returns the available books on the library(their isbns,titles,author names,pubname,categoryName and the Number of copies available.

My query involves the following relations (with bold are the primary keys) :

Book  (ISBN, title,pubYear,numpages, pubName ) pubName FK to publisher

copies  (ISBN, copyNr, shelf ) ISBN FK to Book

borrows (memberID, ISBN, copyNr, date_of_borrowing, date_of_return ) memberID FK to member, ISBN FK to Books, (ISBN, copyNr) FK to copies

belongs_to  (ISBN, categoryName ) SBN FK to Book, categoryName FK to category

written_by  (ISBN, authID ) ISBN FK to Book, authID FK to author

author  (authID, AFirst, ALast, Abirthdate )

My approach is the following:

SELECT b.isbn, b.title,a.ALast, a.ALast, b.pubName, be.categoryName , COUNT(b.isbn) as Number_of_copies_available
FROM copies as c
INNER JOIN book as b ON c.isbn = b.isbn
INNER JOIN belong_to as be ON b.isbn = be.isbn
INNER JOIN written_by as w ON w.isbn=b.isbn
INNER JOIN author as a ON a.authID = w.authID
WHERE (c.isbn,c.copyNr) NOT IN (SELECT isbn, copyNr FROM borrows)
GROUP BY b.isbn
ORDER BY be.categoryName

The problem is that I get duplicate tuples on the result before grouping which leads to more items per group.

For instance for some books I get the attribute value of Number_of_copies_available double than expected.

If I don't JOIN with the relations "written_by" and "author" then the result is correct. However, I also want to appear the author names in the result. What's my mistake? Thanks in advance!

Body:
Don't join the tables directly when they are not perfectly related. A book's category is not really related to a book's author. With two authors and two categories, which author would you combine with which category? It makes no sense to combine them (and possibly end up with all combinations). So, aggregate first and then join the aggregates. Here is an example:

select
  b.isbn,
  b.title,
  b.pubName,
  aut.authors,
  cat.categories,
  cop.total - coalesce(bor.total, 0) as available
from book b
join
(
  select w.isbn, group_concat(a.alast) as authors
  from  written_by w
  join author a ON a.authID = w.authID
  group by w.isbn
) aut ON aut.isbn = b.isbn
join
(
  select isbn, group_concat(categoryname) as categories
  from  belong_to
  group by isbn
) cat ON cat.isbn = b.isbn
join
(
  select isbn, count(*) as total
  from  copies
  group by isbn
) cop ON cop.isbn = b.isbn
left join
(
  select isbn, count(*) as total
  from  borrows
  where date_of_return > current_date
  group by isbn
) bor ON bor.isbn = b.isbn
order by b.isbn;


问:
How do I iterate over a range of numbers defined by variables in Bash?
How do I iterate over a range of numbers in Bash when the range is given by a variable?

I know I can do this (called "sequence expression" in the Bash documentation):

```<code> for i in {1..5}; do echo $i; done
</code>```

Which gives:

1

  2

  3

  4

  5

Yet, how can I replace either of the range endpoints with a variable? This doesn't work:

```<code>END=5
for i in {1..$END}; do echo $i; done
</code>```

Which prints:

{1..5}

答:
There are many ways to do this, however the ones I prefer is given below

Using seq

Synopsis from man seq

```<code>$ seq [-w] [-f format] [-s string] [-t string] [first [incr]] last
</code>```

Syntax

Full command

seq first incr last

first is starting number in the sequence [is optional, by default:1]

incr is increment [is optional, by default:1]

last is the last number in the sequence

Example:

```<code>$ seq 1 2 10
1 3 5 7 9
</code>```

Only with first and last:

```<code>$ seq 1 5
1 2 3 4 5
</code>```

Only with last:

```<code>$ seq 5
1 2 3 4 5
</code>```

Using {first..last..incr}

Here first and last are mandatory and incr is optional

Using just first and last

```<code>$ echo {1..5}
1 2 3 4 5
</code>```

Using incr

```<code>$ echo {1..10..2}
1 3 5 7 9
</code>```

You can use this even for characters like below

```<code>$ echo {a..z}
a b c d e f g h i j k l m n o p q r s t u v w x y z
</code>```

OwnerDisplayName: eschercycle

Body:
How do I iterate over a range of numbers in Bash when the range is given by a variable?

I know I can do this (called "sequence expression" in the Bash documentation):

 for i in {1..5}; do echo $i; done

Which gives:

1

  2

  3

  4

  5

Yet, how can I replace either of the range endpoints with a variable? This doesn't work:

END=5
for i in {1..$END}; do echo $i; done

Which prints:

{1..5}

LastEditorDisplayName: Greg Hewgill

Body:
There are many ways to do this, however the ones I prefer is given below

Using seq

Synopsis from man seq

$ seq [-w] [-f format] [-s string] [-t string] [first [incr]] last

Syntax

Full command

seq first incr last

first is starting number in the sequence [is optional, by default:1]

incr is increment [is optional, by default:1]

last is the last number in the sequence

Example:

$ seq 1 2 10
1 3 5 7 9

Only with first and last:

$ seq 1 5
1 2 3 4 5

Only with last:

$ seq 5
1 2 3 4 5

Using {first..last..incr}

Here first and last are mandatory and incr is optional

Using just first and last

$ echo {1..5}
1 2 3 4 5

Using incr

$ echo {1..10..2}
1 3 5 7 9

You can use this even for characters like below

$ echo {a..z}
a b c d e f g h i j k l m n o p q r s t u v w x y z


问:
Compare two html files
Hi
I want to compare code of two HTML files. How do I achieve this?</p>

答:
Dynamically or just in an editor?

On Windows, use Winmerge. On a Mac, use TextWrangler.

Title: Compare two html files


问:
Construct DFA for the following language on Σ={a,b}
Construct DFA for the following language on Σ={a,b}.

L={w:|w|mod 4=0}

i have the following:

答:
The DFA you made is correct;
As language strings should have length(4 or multiple of 4);
and what you made is correct; as it is also accepting when length gets minimum 4; on final state if again you apply input;
it again goes away from final state until length become multiple of 4.


问:
Why does casting the result from a controller in a unit test return null?
I am in the process of equipping my sources with unit tests, but I made a mistake here.
What can I do at this point if the value is really "null"?
Is that even possible?

I have tried to figure out how others solve it but with the following thread I don't quite get along.
C# in my unit test i want to see if it is null or an empty string

--> If I use IsNullOrEmpty() like in the example above, how does that look like in my code snippet?

This is the affected area in my unit test:

```<code>Assert.That(deviceInfo.SerialNumber == null);
</code>```

The following error message appears:

Message: System.ArgumentNullException : Value cannot be null. Parameter name: source

Edit - This is what my environment looks like:

Here is the class with the DeviceDetails object which has all information in the body.

```<code>public virtual IActionResult DevicesIdGet([FromRoute][Required]string id)
{
    var device = _deviceManager.GetDevice(id);

    if (device == null)
        return NotFound();

    var deviceDetails = new DeviceDetails
    {
        IsOnline = device.IsOnline(),
        SerialNumber = null
    };

    return Ok(deviceDetails);
}
</code>```

Unit-Test

```<code> private Device _testDevice;

 [SetUp]
 public void SetUp()
    {
        _testDevice = new Device
        {
            Id = 1,
            DeviceType = 1,
            DisplayName = "TestDevice",
            IpAddress = IPAddress.Parse("127.0.0.1"),
            IpAddressString = "127.0.0.1"
        };
    }

[Test]
public void If_DeviceIsAvailable_Then_DeviceIdIsSelected()
{
    // ARRANGE
    var deviceManagerMock = new Mock<IDeviceManager>();
    deviceManagerMock.Setup(manager => manager.GetDevices(false))
        .Returns(new List<Device>
                 {
                     _testDevice
                 })
        .Verifiable();

    var subject = new DevicesApiController(deviceManagerMock.Object);

    // ACT
    var result = subject.DevicesIdGet("1");

    // ASSERT
    var deviceInfos = result as IEnumerable<DeviceDetails>;
    var deviceInfo = deviceInfos.Single();
    Assert.That(deviceInfo.IsOnline == true);
    Assert.That(deviceInfo.SerialNumber == null);
}
</code>```

答:
Your controller calls this method of IDeviceManager, which is not mocked:

```<code>var device = _deviceManager.GetDevice(id);
</code>```

Not mocking that method is one cause for the exception. But if you fix it and mock that method (using Setup to specify a result on the mock) you'll still get the same exception. More on that later. First, here's the path your code is going down:

Because it's not mocked, the return value from calling GetDevice on the mock will be null, so this happens:

```<code>if (device == null)
    return NotFound();
</code>```

The result of the method is a NotFound result.

Then, this is happening in the unit test:

```<code>var deviceInfos = result as IEnumerable<DeviceDetails>;
var deviceInfo = deviceInfos.Single();
</code>```

result is a NotFoundResult. result as IEnumerable<DeviceDetails> returns null.

So you're effectively doing this:

```<code>IEnumerable<DeviceDetails> deviceInfos = null;
var deviceInfo = deviceInfos.Single();
</code>```

The source parameter passed to the Single method is null, hence the exception.

If you mock GetDevice, you'll still get the same error for almost exactly the same reason. Now your code will return an OkObjectResult instead of a NotFoundResult. You'll try to cast that as IEnumerable<DeviceDetails>, it will still be null, and you'll get the same exception.

What you need is to get the value from the OkObjectResult, like this:

```<code>var actionResult = subject.DevicesIdGet("1") as OkObjectResult;
var deviceInfos = actionResult.Value as IEnumerable<DeviceDetails> ;
var deviceInfo = deviceInfos.Single();
</code>```

Body:
I am in the process of equipping my sources with unit tests, but I made a mistake here.
What can I do at this point if the value is really "null"?
Is that even possible?

I have tried to figure out how others solve it but with the following thread I don't quite get along.
C# in my unit test i want to see if it is null or an empty string

--> If I use IsNullOrEmpty() like in the example above, how does that look like in my code snippet?

This is the affected area in my unit test:

Assert.That(deviceInfo.SerialNumber == null);

The following error message appears:

Message: System.ArgumentNullException : Value cannot be null. Parameter name: source

Edit - This is what my environment looks like:

Here is the class with the DeviceDetails object which has all information in the body.

public virtual IActionResult DevicesIdGet([FromRoute][Required]string id)
{
    var device = _deviceManager.GetDevice(id);

    if (device == null)
        return NotFound();

    var deviceDetails = new DeviceDetails
    {
        IsOnline = device.IsOnline(),
        SerialNumber = null
    };

    return Ok(deviceDetails);
}

Unit-Test

 private Device _testDevice;

 [SetUp]
 public void SetUp()
    {
        _testDevice = new Device
        {
            Id = 1,
            DeviceType = 1,
            DisplayName = "TestDevice",
            IpAddress = IPAddress.Parse("127.0.0.1"),
            IpAddressString = "127.0.0.1"
        };
    }

[Test]
public void If_DeviceIsAvailable_Then_DeviceIdIsSelected()
{
    // ARRANGE
    var deviceManagerMock = new Mock<IDeviceManager>();
    deviceManagerMock.Setup(manager => manager.GetDevices(false))
        .Returns(new List<Device>
                 {
                     _testDevice
                 })
        .Verifiable();

    var subject = new DevicesApiController(deviceManagerMock.Object);

    // ACT
    var result = subject.DevicesIdGet("1");

    // ASSERT
    var deviceInfos = result as IEnumerable<DeviceDetails>;
    var deviceInfo = deviceInfos.Single();
    Assert.That(deviceInfo.IsOnline == true);
    Assert.That(deviceInfo.SerialNumber == null);
}

Body:
Your controller calls this method of IDeviceManager, which is not mocked:

var device = _deviceManager.GetDevice(id);

Not mocking that method is one cause for the exception. But if you fix it and mock that method (using Setup to specify a result on the mock) you'll still get the same exception. More on that later. First, here's the path your code is going down:

Because it's not mocked, the return value from calling GetDevice on the mock will be null, so this happens:

if (device == null)
    return NotFound();

The result of the method is a NotFound result.

Then, this is happening in the unit test:

var deviceInfos = result as IEnumerable<DeviceDetails>;
var deviceInfo = deviceInfos.Single();

result is a NotFoundResult. result as IEnumerable<DeviceDetails> returns null.

So you're effectively doing this:

IEnumerable<DeviceDetails> deviceInfos = null;
var deviceInfo = deviceInfos.Single();

The source parameter passed to the Single method is null, hence the exception.

If you mock GetDevice, you'll still get the same error for almost exactly the same reason. Now your code will return an OkObjectResult instead of a NotFoundResult. You'll try to cast that as IEnumerable<DeviceDetails>, it will still be null, and you'll get the same exception.

What you need is to get the value from the OkObjectResult, like this:

var actionResult = subject.DevicesIdGet("1") as OkObjectResult;
var deviceInfos = actionResult.Value as IEnumerable<DeviceDetails> ;
var deviceInfo = deviceInfos.Single();


问:
Coq: unfold tricky notations
Suppose I want to convert ~X to X -> False using unfold tactic.

I have two options how to do it: either by unfold not or by unfold "~".
The second way is apparently more convenient since there is no need to remember what definition name stays behind the notation.

However I haven't managed to do it in more complex cases:

```<code>Definition assn_sub X a P : Assertion :=
  fun (st : state) =>
    P (X !-> aeval st a ; st).

Notation "P [ X |-> a ]" := (assn_sub X a P)
  (at level 10, X at next level).
</code>```

and I want to unfold it in expression

```<code>((fun st0 : state => st0 Y = st0 X + st0 Z) [Z |-> Y - X]) st
</code>```

without using assn_sub. Is it possible?

答:
Hard to say without a reproducible example, but Ssreflect's unfold should work:</p>

```<code>rewrite /(_[_|->_]).
</code>```

Title: Coq: unfold tricky notations

Body:
Suppose I want to convert ~X to X -> False using unfold tactic.

I have two options how to do it: either by unfold not or by unfold "~".
The second way is apparently more convenient since there is no need to remember what definition name stays behind the notation.

However I haven't managed to do it in more complex cases:

Definition assn_sub X a P : Assertion :=
  fun (st : state) =>
    P (X !-> aeval st a ; st).

Notation "P [ X |-> a ]" := (assn_sub X a P)
  (at level 10, X at next level).

and I want to unfold it in expression

((fun st0 : state => st0 Y = st0 X + st0 Z) [Z |-> Y - X]) st

without using assn_sub. Is it possible?

Body:
Hard to say without a reproducible example, but Ssreflect's unfold should work:

rewrite /(_[_|->_]).


问:
Checking reciever account activity in PayPal using API
I plan to use mass payments in PayPal.

https://developer.paypal.com/docs/api/payments.payouts-batch/v1/

Can I ask you a question: before creating payout I want to check reciever account (Determines whether the specified PayPal account's status is verified or unverified). I want to exclude the possibility of an error in payment.

I see like this scenario.

Client: Hi, can I add funds to my account?

I: What is your ID (Email\Phone)?

Client: Hmm.. hello@world.com

I: Ok. Pls wait a few second. I will check your account

I: (I search client account in PayPal) Yes. Your account is valid! How much do you want to add funds?

Client: 5 dollars.

I: Ok. Create payout!

Is it possible to make a similar scenario in API?

答:
I found solution. Method GetVerifiedStatus.

https://developer.paypal.com/docs/classic/adaptive-accounts/integration-guide/ACGettingStarted/?mark=GetVerifiedStatus#adaptive-accounts-methods


问:
Passing Python objects between Tasks in Luigi?
I was coding my first project in Python 3.6 using Spotify's Luigi to arrange some Natural Language Processing Tasks in a pipeline.

I noticed that the output() function of a Task class always returns some kind of Target object, which is just some file somewhere, be it local or remote. Because my Tasks produce more complex data structures like parse trees, it's pretty awkward for me to write them into files as strings and read them again after.

Therefore I would like to ask if there is any possibility to pass Python objects between the tasks within a pipeline?

答:
There is also other - still a bit hacky - way to achieve what you're trying to do with target instead of parameters.

There is a special MockFile target in luigi.mock that allows you to store it's "file" in memory.

It's api is similar to other Target inheriting classes, so you'll have to open, read and write to it. Suddenly it only supports string input, so you're still need to serialise your object (that's due to sending this data through pipe between processes). See the following example (yaml serialisation):

import yaml
from luigi import Task

class TaskA(Task):
    def output(self):
        return MockFile('whatever')

    def run(self):
        object_to_send = yaml.dump({"example": "dict"})

        _out = self.output().open('r')
        _out.write(object_to_send)
        _out.close()

class TaskB(Task):
    def requires(self):
        return TaskA()

    def run(self):
        _in = self.input().read('r')
        serialised = _in.read()
        deserialised = yaml.load(serialised)
        print(deserialised)
```

Be aware, that serialising big objects might take a lot of time.

Body:
There is also other - still a bit hacky - way to achieve what you're trying to do with target instead of parameters.

There is a special MockFile target in luigi.mock that allows you to store it's "file" in memory.

It's api is similar to other Target inheriting classes, so you'll have to open, read and write to it. Suddenly it only supports string input, so you're still need to serialise your object (that's due to sending this data through pipe between processes). See the following example (yaml serialisation):

import yaml
from luigi import Task

class TaskA(Task):
    def output(self):
        return MockFile('whatever')

    def run(self):
        object_to_send = yaml.dump({"example": "dict"})

        _out = self.output().open('r')
        _out.write(object_to_send)
        _out.close()

class TaskB(Task):
    def requires(self):
        return TaskA()

    def run(self):
        _in = self.input().read('r')
        serialised = _in.read()
        deserialised = yaml.load(serialised)
        print(deserialised)

Be aware, that serialising big objects might take a lot of time.


问:
WSO2 ESB endpoint using cached backend name
We can see calls going from ESB to backend system old Ip instead of the new IP. For example

Wso2 ESB Endpoint configured as: somebackend.com

If we do nslookup to see the IP of the system.

```<code>$nslookup somebackend.com
10.20.30.40
</code>```

If we trace the traffic from ESB using tcpdump with host as "somebackend.com" then calls are going to old IP of the system (somebackend.com) i,e, 50.60.70.80 instead of new IP 10.20.30.40

Is the name/Ip/DNS Name is cached anywhere in the WSo2 ESB.?
Any suggestions, please resolve this.
Below is the endpoint file configured.

```<code><endpoint xmlns="http://ws.apache.org/ns/synapse"
          name="conf//endpoint/endpointname">
    <property name="System-Name" value="Backend" scope="default" type="STRING"/>
   <http uri-template="somebackend.com"
         method="get">
    <suspendOnFailure>
           <errorCodes>-1</errorCodes>
           <initialDuration>0</initialDuration>
           <progressionFactor>1.0</progressionFactor>
           <maximumDuration>0</maximumDuration>
      </suspendOnFailure>
      <markForSuspension>
         <errorCodes>-1</errorCodes>
      </markForSuspension>
   </http>
</endpoint>
</code>```

答: Can you check in your /etc/hosts file whether you have wrongly mapped the old IP to the domain name (somebackend.com)? It might be a possibility because AFAIK ESB does not cache IP. Maybe restarting the ESB server would also help.</p>

Body:
We can see calls going from ESB to backend system old Ip instead of the new IP. For example

Wso2 ESB Endpoint configured as: somebackend.com

If we do nslookup to see the IP of the system.

$nslookup somebackend.com
10.20.30.40

If we trace the traffic from ESB using tcpdump with host as "somebackend.com" then calls are going to old IP of the system (somebackend.com) i,e, 50.60.70.80 instead of new IP 10.20.30.40

Is the name/Ip/DNS Name is cached anywhere in the WSo2 ESB.?
Any suggestions, please resolve this.
Below is the endpoint file configured.

<endpoint xmlns="http://ws.apache.org/ns/synapse"
          name="conf//endpoint/endpointname">
    <property name="System-Name" value="Backend" scope="default" type="STRING"/>
   <http uri-template="somebackend.com"
         method="get">
    <suspendOnFailure>
           <errorCodes>-1</errorCodes>
           <initialDuration>0</initialDuration>
           <progressionFactor>1.0</progressionFactor>
           <maximumDuration>0</maximumDuration>
      </suspendOnFailure>
      <markForSuspension>
         <errorCodes>-1</errorCodes>
      </markForSuspension>
   </http>
</endpoint>


问:
jQuery ajax ".done" callbacks not firing
The following jQuery code (postData) posts data to a simple php file and on completion calls sendReply or sendFail.  I have a console.log in the postData function and the sendReply and sendFail callbacks also have console.log.  The first console.log shows in the FF dev console, the callbacks do not.

The php file is very simple because I'm new to posting to php with jQuery and I want to test the callback function before I go on.

Here is the jQuery:

```<code><form>

// form elements here, omitted for brevity
 // the jQuery is called from a button click event

<div class="btn_div">
<button class="btn_joinnow" id="btn_submit" style="color:rgb(255,255,255)" onClick="postData(event)">Click to submit data</button></div>

<script type="text/javascript">
function postData(event) {
    event.preventDefault();
    var datastring = $("#echo_test").serialize();
    console.log("Okay, I'm starting")
    return $.ajax({
       type: "POST",
       url: "echo_test.php",
       data: {post: datastring}
    });
}

function sendReply() {
    console.log("Okay, I'm done"); }

function sendFail() {
    console.log("Okay, I failed"); }

postData().done(sendReply, sendFail);

</script>

</form>
</code>```

I could chain the callbacks, but then event.preventDefault does not work:

```<code><script type="text/javascript">
function postData(event) {
    event.preventDefault();
    var datastring = $("#echo_test").serialize();
    console.log("Okay, I'm starting")
    return $.ajax({
       type: "POST",
       url: "echo_test.php",
       data: {post: datastring}
    });.done(function() {
    console.log("Okay, I'm done");
    }).fail(function() {
    console.log("Okay, I failed");
});
}
</code>```

Here is the ultra-simple php:

```<code><?php
echo "Hello world";
?>
</code>```

Why doesn't eithier callback function execute on completion?  Is it the jQuery or the php function?  The Firefox dev console shows this about the php:

```<code>XML Parsing Error: no root element found Location: (file location) Line Number 4, Column 1:
</code>```

I am working on my local (Windows) machine, not on my cloud server.  Is that the entire problem?

Thanks very much for any insights.

答:
Here is a working version with some small tweaks since the php you have does not care about the data you post.

To keep it as a post, I changed it to jsonplaceholder and after clicking you can see the return response in the console.

<form>

<button href="google.com"  class="btn_joinnow" id="btn_submit" style="color:rgb(255,255,255)">Click to submit data

    </button>

</form>
```

<p>I added a link to google to make sure the preventDefault does work. </p>

```$("#btn_submit").on("click", function (event) {
    event.preventDefault();
    var datastring = JSON.stringify({
        title: 'foo',
        body: 'bar',
        userId: 1
    });
    console.log("Okay, I'm starting");
    return $.ajax({
        type: "POST",
        url: "https://jsonplaceholder.typicode.com/posts",
        data: {post: datastring},
        success: function (response) {
            console.log(response);
        },
        error: function (error) {
            console.log("Okay, I failed" + error);
        }
    });
});
```

If you want it simpler, with a GET, here is a codepen.

Body:
The following jQuery code (postData) posts data to a simple php file and on completion calls sendReply or sendFail.  I have a console.log in the postData function and the sendReply and sendFail callbacks also have console.log.  The first console.log shows in the FF dev console, the callbacks do not.

The php file is very simple because I'm new to posting to php with jQuery and I want to test the callback function before I go on.

Here is the jQuery:

<form>

// form elements here, omitted for brevity
 // the jQuery is called from a button click event

<button class="btn_joinnow" id="btn_submit" style="color:rgb(255,255,255)" onClick="postData(event)">Click to submit data</button>

</form>

I could chain the callbacks, but then event.preventDefault does not work:

Body:
Here is a working version with some small tweaks since the php you have does not care about the data you post.

To keep it as a post, I changed it to jsonplaceholder and after clicking you can see the return response in the console.

<form>
<div class="btn_div">
<button href="google.com"  class="btn_joinnow" id="btn_submit" style="color:rgb(255,255,255)">Click to submit data

    </button></div>
</form>

I added a link to google to make sure the preventDefault does work.

$("#btn_submit").on("click", function (event) {
    event.preventDefault();
    var datastring = JSON.stringify({
        title: 'foo',
        body: 'bar',
        userId: 1
    });
    console.log("Okay, I'm starting");
    return $.ajax({
        type: "POST",
        url: "https://jsonplaceholder.typicode.com/posts",
        data: {post: datastring},
        success: function (response) {
            console.log(response);
        },
        error: function (error) {
            console.log("Okay, I failed" + error);
        }
    });
});

If you want it simpler, with a GET, here is a codepen.


问:
FOR SYSTEM_TIME ALL with ORDER BY
On top of exposing all temporal tables as traditional tables, to enable auditing I'm exposing all temporal tables separately with the FOR SYSTEM_TIME ALL hint. This is easier to consume as the current record is also found in the resultset then.

However, an ORDER BY on my Account table in conjunction with the temporal FOR SYSTEM_TIME ALL hint is causing it to spill data into tempdb. This is a real performance killer. I need the ORDER BY for pagination. I've also tried a UNION of both the temporal table and the history table and applying an ORDER BY on that but this naturally leads to the same performance problem. Can anything be done to adress this problem?

Below you will find the actual execution plans. Indexes spanning across the primary key and the period columns are present on both tables as can be seen.

Just FOR SYTEM_TIME ALL:

Just ORDER BY:

FOR SYSTEM_TIME ALL with ORDER BY:

答: In my opinion the problem is not spilling data to disc. It's an effect, not the root cause. In the first query there is no order by, so SQL Server could give you any row. In the second query there was an index on Account that supported the sort. In the last query the data was spilled, because on history table there is no index on ID, so SQL Server had to read all the history records to be able to give you a record sorted by ID. I think that craeting a nonclustered index on the history table on ID might help (you cannot create a clustered index on History, becuase there is already on Account).

Title: FOR SYSTEM_TIME ALL with ORDER BY


问:
FITS with WCS to healpy
Assume I have a FITS file that contains a WCS header, so that I can do:

#import healpy as hp
#import astropy.io.fits as pyfits
#from astropy.wcs import WCS

listofhdus = pyfits.open(FITS)
wcs = WCS(listofhdus[0].header)
```

<p>listofhdus[0].data would contain a 2D numpy array (NY times NX) which correspond to a small portion of the full sky in Galactic coordinates.</p>

<p>What would be the best way to convert that 2d numpy into healpix format, known the WCS, if I want to include overlay that skymap into the following scatter plot with Mollweide projection?</p>

<pre class="lang-py prettyprint-override"><code>NPIX = hp.nside2npix(512)
m = 0*np.arange(NPIX)
hp.mollview(m, title="test")
hp.projscatter(longitude, latitude, lonlat=True,
               coord='G',marker='+',color='black')
hp.graticule()
</code>```

where longitude and latitude are two 2D arrays (generated with numpy.meshgrid) with Galactic coordinates of some astrophysical sources I am interested on?.

I guess I could try to translate my healpix pixels into coordinates, somehow match them with the ones available in my skymap and interpolate the values from there, but there must be something easier, more elegant and precise, right?

答:
You should check out the reproject, which takes care of all your reprojection needs to/from HEALPix.

In particular, check out reproject_to_healpix.

If you need a better control of your interpolation routine (using a convolution-based gridder, ensuring flux conservation, performance), I'd suggest you check out cygrid, which has some examples for HEALPix as well.

Title: FITS with WCS to healpy

Body:
Assume I have a FITS file that contains a WCS header, so that I can do:

#import healpy as hp
#import astropy.io.fits as pyfits
#from astropy.wcs import WCS

listofhdus = pyfits.open(FITS)
wcs = WCS(listofhdus[0].header)

listofhdus[0].data would contain a 2D numpy array (NY times NX) which correspond to a small portion of the full sky in Galactic coordinates.

What would be the best way to convert that 2d numpy into healpix format, known the WCS, if I want to include overlay that skymap into the following scatter plot with Mollweide projection?

NPIX = hp.nside2npix(512)
m = 0*np.arange(NPIX)
hp.mollview(m, title="test")
hp.projscatter(longitude, latitude, lonlat=True,
               coord='G',marker='+',color='black')
hp.graticule()

where longitude and latitude are two 2D arrays (generated with numpy.meshgrid) with Galactic coordinates of some astrophysical sources I am interested on?.

I guess I could try to translate my healpix pixels into coordinates, somehow match them with the ones available in my skymap and interpolate the values from there, but there must be something easier, more elegant and precise, right?


问:
Can't create custom colorswatch
I'm trying to use a custom color theme for an app.

Here is the main.dart file

```<code>import 'package:flutter/material.dart';

import 'pages/HomePage.dart';

void main() => runApp(new MyApp());

class MyApp extends StatelessWidget {
  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return new MaterialApp(
      title: 'Flutter Demo',
      debugShowCheckedModeBanner: false,
      theme: new ThemeData(
        // This is the theme of your application.
        //
        // Try running your application with "flutter run". You'll see the
        // application has a blue toolbar. Then, without quitting the app, try
        // changing the primarySwatch below to Colors.green and then invoke
        // "hot reload" (press "r" in the console where you ran "flutter run",
        // or press Run > Flutter Hot Reload in IntelliJ). Notice that the
        // counter didn't reset back to zero; the application is not restarted.
        primarySwatch: MaterialColor(0xFF990000, {
          50: Color(0xFF990001),
          300: Color(0xFF990002),
          900: Color(0xFF990003)
        }),
      ),
      home: new HomePage(),
    );
  }
}
</code>```

I get the following error:

flutter: ══╡ EXCEPTION CAUGHT BY WIDGETS LIBRARY ╞═══════════════════════════════════════════════════════════
flutter: The following NoSuchMethodError was thrown building MyApp(dirty):
flutter: The method 'computeLuminance' was called on null.
flutter: Receiver: null
flutter: Tried calling: computeLuminance()

What I'm I doing wrong?

答:
You need to create one MaterialColor object as a whole, providing all color values for the ten different luminance values. Please consider using this solution for a better approach.

Flutter: Creating a custom color swatch for MaterialColor

Title: Can't create custom colorswatch

Body:
I'm trying to use a custom color theme for an app.

Here is the main.dart file

import 'package:flutter/material.dart';

import 'pages/HomePage.dart';

void main() => runApp(new MyApp());

class MyApp extends StatelessWidget {
  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return new MaterialApp(
      title: 'Flutter Demo',
      debugShowCheckedModeBanner: false,
      theme: new ThemeData(
        // This is the theme of your application.
        //
        // Try running your application with "flutter run". You'll see the
        // application has a blue toolbar. Then, without quitting the app, try
        // changing the primarySwatch below to Colors.green and then invoke
        // "hot reload" (press "r" in the console where you ran "flutter run",
        // or press Run > Flutter Hot Reload in IntelliJ). Notice that the
        // counter didn't reset back to zero; the application is not restarted.
        primarySwatch: MaterialColor(0xFF990000, {
          50: Color(0xFF990001),
          300: Color(0xFF990002),
          900: Color(0xFF990003)
        }),
      ),
      home: new HomePage(),
    );
  }
}

I get the following error:

flutter: ══╡ EXCEPTION CAUGHT BY WIDGETS LIBRARY ╞═══════════════════════════════════════════════════════════
flutter: The following NoSuchMethodError was thrown building MyApp(dirty):
flutter: The method 'computeLuminance' was called on null.
flutter: Receiver: null
flutter: Tried calling: computeLuminance()

What I'm I doing wrong?


Title: Python Pandas Column Sort by Count

Body:
I am still new to python and pandas. I am trying to figure out how to sort the data according to each column's row count.
The count of the row excluding NaN is:

np.count_nonzero(~np.isnan(df.iloc[:, j]), where j is the column number

import pandas as pd
x = np.nan
data = {"A": [22133.97, 22151.06, 22836.96, 22035.42, 23306.39, 23881.29, 23206.37], "B": [95.924, 107.1005, 123.5775, 107.8946, x, x, x], "C": [74.169, 74.075, 77.564, 76.338, 79.356, 81.666, x], "D": [36.205, 35.435, 36.542, 34.424, 37.457, x, x], "E": [68.048, 65.554, 68.093, 68.37, 74.233, 77.095, 75.156]}
dates = pd.date_range('1/1/2000', periods = 7)
df = pd.DataFrame(data, index = dates)

The dataset now looks like this:

                   A         B       C       D       E
2000-01-01  22133.97   95.9240  74.169  36.205  68.048
2000-01-02  22151.06  107.1005  74.075  35.435  65.554
2000-01-03  22836.96  123.5775  77.564  36.542  68.093
2000-01-04  22035.42  107.8946  76.338  34.424  68.370
2000-01-05  23306.39       NaN  79.356  37.457  74.233
2000-01-06  23881.29       NaN  81.666     NaN  77.095
2000-01-07  23206.37       NaN     NaN     NaN  75.156

Is there any in-built pandas function to sort the columns such that it looks like the following (i.e column A has 7 rows, so it remains, then searches for the next most column which is in E and it shifts it forward and places it beside Column A so on and so forth).

                   A       E       C       D         B
2000-01-01  22133.97  68.048  74.169  36.205   95.9240
2000-01-02  22151.06  65.554  74.075  35.435  107.1005
2000-01-03  22836.96  68.093  77.564  36.542  123.5775
2000-01-04  22035.42  68.370  76.338  34.424  107.8946
2000-01-05  23306.39  74.233  79.356  37.457       NaN
2000-01-06  23881.29  77.095  81.666     NaN       NaN
2000-01-07  23206.37  75.156     NaN     NaN       NaN

Would appreciate any help. Thanks!

Body:
I think this code will work.

df=df[df.isna().sum().sort_values().keys()]

Output:

>>>df[df.isna().sum().sort_values().keys()]
                   A       E       C       D         B
2000-01-01  22133.97  68.048  74.169  36.205   95.9240
2000-01-02  22151.06  65.554  74.075  35.435  107.1005
2000-01-03  22836.96  68.093  77.564  36.542  123.5775
2000-01-04  22035.42  68.370  76.338  34.424  107.8946
2000-01-05  23306.39  74.233  79.356  37.457       NaN
2000-01-06  23881.29  77.095  81.666     NaN       NaN
2000-01-07  23206.37  75.156     NaN     NaN       NaN


问:
How to check the date is empty using python?
The startdate and enddate values are coming from jenkin to lambda.
In this code I am trying to get value using event["start_date"] and event["end_date"] and it's working fine, but if startdate and enddate are not available then the code should automatically take today's and yesterday's date.

I am new to python, can someone help here?

I tried as below but I am getting error.
If I will mention 'startdate and enddate is none' and set both values as none then it's working but I need to implement mentioned in code as I am using AWS lambda with python.

```<code>import datetime
from datetime import timedelta

dateformat = "%Y-%m-%d"
startdate = datetime.datetime.strptime("", dateformat)
enddate = datetime.datetime.strptime("", dateformat)
# Both startdate and enddate values come from Jenkin to lambda  -
event["start_date"] and event["end_date"]
if (startdate and enddate) == "":
    startdate = datetime.date.today()
    enddate = startdate - datetime.timedelta(days = 1)
    print('yesterday : ', enddate)
    print('Today : ', startdate)
</code>```

Current Error Output:

```<code>Traceback (most recent call last):
  File "variable_value_change.py", line 10, in <module>
    startdate = datetime.datetime.strptime("", dateformat)
  File "C:\Users\336635743\AppData\Local\Programs\Python\Python37-32\lib\_strptime.py", line 577, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "C:\Users\336635743\AppData\Local\Programs\Python\Python37-32\lib\_strptime.py", line 359, in _strptime
   (data_string, format))
ValueError: time data '' does not match format '%Y-%m-%d'
</code>```

答:
Use a try-except block.

Ex:

```<code>try:
    startdate = datetime.datetime.strptime(event["start_date"], dateformat)
    enddate = datetime.datetime.strptime(event["end_Date"], dateformat)
except:
    startdate = datetime.date.today()
    enddate = startdate - datetime.timedelta(days = 1)
</code>```

Body:
The startdate and enddate values are coming from jenkin to lambda.
In this code I am trying to get value using event["start_date"] and event["end_date"] and it's working fine, but if startdate and enddate are not available then the code should automatically take today's and yesterday's date.

I am new to python, can someone help here?

I tried as below but I am getting error.
If I will mention 'startdate and enddate is none' and set both values as none then it's working but I need to implement mentioned in code as I am using AWS lambda with python.

import datetime
from datetime import timedelta

dateformat = "%Y-%m-%d"
startdate = datetime.datetime.strptime("", dateformat)
enddate = datetime.datetime.strptime("", dateformat)
# Both startdate and enddate values come from Jenkin to lambda  -
event["start_date"] and event["end_date"]
if (startdate and enddate) == "":
    startdate = datetime.date.today()
    enddate = startdate - datetime.timedelta(days = 1)
    print('yesterday : ', enddate)
    print('Today : ', startdate)

Current Error Output:

Traceback (most recent call last):
  File "variable_value_change.py", line 10, in <module>
    startdate = datetime.datetime.strptime("", dateformat)
  File "C:\Users\336635743\AppData\Local\Programs\Python\Python37-32\lib\_strptime.py", line 577, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "C:\Users\336635743\AppData\Local\Programs\Python\Python37-32\lib\_strptime.py", line 359, in _strptime
   (data_string, format))
ValueError: time data '' does not match format '%Y-%m-%d'

Body:
Use a try-except block.

Ex:

try:
    startdate = datetime.datetime.strptime(event["start_date"], dateformat)
    enddate = datetime.datetime.strptime(event["end_Date"], dateformat)
except:
    startdate = datetime.date.today()
    enddate = startdate - datetime.timedelta(days = 1)


问:
VB6: Get hWnd of a child control inside a 3rd Party control
I have a vb6 third party UpDown Control (let it be ControlX), with UISpy i could see that ControlX has 2 controls inside, one is a "ThunderRT6TextBox" the other is a "UpDown20WndClass".

I am drawing a border around ControlX. I am using the ControlX hWnd, and i draw the border like this:

```<code>   hdc = BeginPaint(hwnd, tPS)

   GetClientRect hwnd, controlXRect

   DrawEdge hdc, controlXRect, BDR_SUNKENOUTER, BF_RECT
</code>```

The problem is that the border is drawn around the ThunderRT6TextBox but not the UpDown20WndClass (maybe the ControlX hWnd returns it's inner ThunderRT6TextBox control hwnd).

I would like to get the ControlX's inner UpDown20WndClass control hWnd, to draw a border around it.

How can i do this?

Thanks in advance.

答: I when get ControlX hwnd it returns the hwnd of the "ThunderRT6TextBox". To get the "UpDown20WndClass" hwnd, i need its parent hwnd (ControlX hwnd).  I used GetParent with "ThunderRT6TextBox" hwnd, and got the common parent hwnd, and then with FindWindowEx i got the "UpDown20WndClass".</p>

Body:
I have a vb6 third party UpDown Control (let it be ControlX), with UISpy i could see that ControlX has 2 controls inside, one is a "ThunderRT6TextBox" the other is a "UpDown20WndClass".

I am drawing a border around ControlX. I am using the ControlX hWnd, and i draw the border like this:

   hdc = BeginPaint(hwnd, tPS)

   GetClientRect hwnd, controlXRect

   DrawEdge hdc, controlXRect, BDR_SUNKENOUTER, BF_RECT

The problem is that the border is drawn around the ThunderRT6TextBox but not the UpDown20WndClass (maybe the ControlX hWnd returns it's inner ThunderRT6TextBox control hwnd).

I would like to get the ControlX's inner UpDown20WndClass control hWnd, to draw a border around it.

How can i do this?

Thanks in advance.


问:
Static notification in Android application [AS3]
A few days ago I've been looking for a solution to using a Push Notification, but I have not found ANE's functional for the problem.

I'm finalizing an application in the ActionScript3 language, and I only found something related to Cloud Messaging, which is something much more advanced than I need.

I try to present a simple Push Notification to the user, static, only with Title and Message. However, I could not find ways to find such a resource.

Does anyone using the ActionScript language have achieved the feat of using Push Notifications without Google's Cloud Messaging feature? Waiting for answers :)

I went after explanatory projects on GitHub, but 60% of ANE's content is cloud-centric or not working. In addition to having some I'm not targeting the AS3 language.

答:
why you don't try local notification?
in this ANE you can display local notifications with only a few lines of code

I have the same project and use this ANE, it works like a charm
https://airnativeextensions.com/extension/com.distriqt.Notifications


问:
VMware Workstation and Device/Credential Guard are not compatible
I have been running VMware for the last year no problems, today I opened it up to start one of my VM and get an error message, see screen shot.

I did follow the link and went through the steps, on step 4 I need to mount a volume using "mountvol".
when I try to mount a volume using mountvol X: \\?\Volume{5593b5bd-0000-0000-0000-c0f373000000}\ it keeps saying The directory is not empty. I even created a partition with 2GB and still the same message.

My Questions:

How can I mount the volume that is not empty even though it is?

Why did this Device/Credential Guard auto enable itself and how can I get rid of it or disable it.

CMD:

答:
If you are someone who maintains an open customized "Run as administrator" command prompt or powershell command line window at all the times you can optionally setup the following aliases / macros to simplify executing the commands mentioned by @gue22 for simply disabling hyper-v hypervisor when needing to use vmware player or workstation and then enabling it again when done.

```<code>doskey hpvEnb = choice /c:yn /cs /d n /t 30 /m "Are you running from elevated command prompt" ^& if not errorlevel 2 ( bcdedit /set hypervisorlaunchtype auto ^& echo.^&echo now reboot to enable hyper-v hypervisor )
doskey hpvDis = choice /c:yn /cs /d n /t 30 /m "Are you running from elevated command prompt" ^& if not errorlevel 2 ( bcdedit /set hypervisorlaunchtype off ^& echo.^&echo now reboot to disable hyper-v hypervisor )
doskey bcdL = bcdedit /enum ^& echo.^&echo now see boot configuration data store {current} boot loader settings
</code>```

With the above in place you just type "hpvenb" [ hypervisor enabled at boot ], "hpvdis" [ hypervisor disabled at boot ] and "bcdl" [ boot configuration devices list ] commands to execute the on, off, list commands.

Body:
If you are someone who maintains an open customized "Run as administrator" command prompt or powershell command line window at all the times you can optionally setup the following aliases / macros to simplify executing the commands mentioned by @gue22 for simply disabling hyper-v hypervisor when needing to use vmware player or workstation and then enabling it again when done.

doskey hpvEnb = choice /c:yn /cs /d n /t 30 /m "Are you running from elevated command prompt" ^& if not errorlevel 2 ( bcdedit /set hypervisorlaunchtype auto ^& echo.^&echo now reboot to enable hyper-v hypervisor )
doskey hpvDis = choice /c:yn /cs /d n /t 30 /m "Are you running from elevated command prompt" ^& if not errorlevel 2 ( bcdedit /set hypervisorlaunchtype off ^& echo.^&echo now reboot to disable hyper-v hypervisor )
doskey bcdL = bcdedit /enum ^& echo.^&echo now see boot configuration data store {current} boot loader settings

With the above in place you just type "hpvenb" [ hypervisor enabled at boot ], "hpvdis" [ hypervisor disabled at boot ] and "bcdl" [ boot configuration devices list ] commands to execute the on, off, list commands.