问:
Aerospike - No improvements in latency on moving to in-memory cluster from on-disk cluster
To begin with, we had an aerospike cluster having 5 nodes of i2.2xlarge type in AWS, which our production fleet of around 200 servers was using to store/retrieve data. The aerospike config of the cluster was as follows -

```<code>service {
        user root
        group root
        paxos-single-replica-limit 1 # Number of nodes where the replica count is automatically reduced to 1.
        pidfile /var/run/aerospike/asd.pid
        service-threads 8
        transaction-queues 8
        transaction-threads-per-queue 4
        fabric-workers 8
        transaction-pending-limit 100
        proto-fd-max 25000
}

logging {
        # Log file must be an absolute path.
        file /var/log/aerospike/aerospike.log {
                context any info
        }
}

network {
        service {
                address any
                port 3000
        }

        heartbeat {
                mode mesh
                port 3002 # Heartbeat port for this node.

                # List one or more other nodes, one ip-address & port per line:
                mesh-seed-address-port <IP> 3002
                mesh-seed-address-port <IP> 3002
                mesh-seed-address-port <IP> 3002
                mesh-seed-address-port <IP> 3002
               # mesh-seed-address-port <IP> 3002
                interval 250
                timeout 10
        }

        fabric {
                port 3001
        }

        info {
                port 3003
        }
}

namespace FC {
        replication-factor 2
        memory-size 7G
        default-ttl 30d # 30 days, use 0 to never expire/evict.

        high-water-disk-pct 80    # How full may the disk become before the server begins eviction
            high-water-memory-pct 70 # Evict non-zero TTL data if capacity exceeds # 70% of 15GB
               stop-writes-pct 90       # Stop writes if capacity exceeds 90% of 15GB

                  storage-engine device {
                  device /dev/xvdb1
                  write-block-size 256K
                      }

}
</code>```

It was properly handling the traffic corresponding to the namespace "FC", with latencies within 14 ms, as shown in the following graph plotted using graphite -

However, on turning on another namespace, with much higher traffic on the same cluster, it started to give a lot of timeouts and higher latencies, as we scaled up the number of servers using the same cluster of 5 nodes (increasing number of servers step by step from 20 to 40 to 60) with the following namespace configuration -

```<code>namespace HEAVYNAMESPACE {
        replication-factor 2
        memory-size 35G
        default-ttl 30d # 30 days, use 0 to never expire/evict.

 high-water-disk-pct 80    # How full may the disk become before the server begins eviction
    high-water-memory-pct 70 # Evict non-zero TTL data if capacity exceeds # 70% of 35GB
   stop-writes-pct 90       # Stop writes if capacity exceeds 90% of 35GB

   storage-engine device {
device /dev/xvdb8
write-block-size 256K
    }

}
</code>```

Following were the observations -

----FC Namespace----

```<code>20 - servers, 6k Write TPS, 16K Read TPS
set latency = 10ms
set timeouts = 1
get latency = 15ms
get timeouts = 3

40 - servers, 12k Write TPS, 17K Read TPS
set latency = 12ms
set timeouts = 1
get latency = 20ms
get timeouts = 5

60 - servers, 17k Write TPS, 18K Read TPS
set latency = 25ms
set timeouts = 5
get latency = 30ms
get timeouts = 10-50 (fluctuating)
</code>```

----HEAVYNAMESPACE----

```<code>20 - del servers, 6k Write TPS, 16K Read TPS
set latency = 7ms
set timeouts = 1
get latency = 5ms
get timeouts = 0
no of keys = 47 million x 2
disk usage = 121 gb
ram usage = 5.62 gb

40 - del servers, 12k Write TPS, 17K Read TPS
set latency = 15ms
set timeouts = 5
get latency = 12ms
get timeouts = 2

60 - del servers, 17k Write TPS, 18K Read TPS
set latency = 25ms
set timeouts = 25-75 (fluctuating)
get latency = 25ms
get timeouts = 2-15 (fluctuating)
</code>```

* Set latency refers to latency in setting aerospike cache keys and similarly get for getting keys.

We had to turn off the namespace "HEAVYNAMESPACE" after reaching 60 servers.

We then started a fresh POC with a cluster having nodes which were r3.4xlarge instances of AWS (find details here https://aws.amazon.com/ec2/instance-types/), with the key difference in aerospike configuration being the usage of memory only for caching, hoping that it would give better performance. Here is the aerospike.conf file -

```<code>service {
        user root
        group root
        paxos-single-replica-limit 1 # Number of nodes where the replica count is automatically reduced to 1.
        pidfile /var/run/aerospike/asd.pid
        service-threads 16
        transaction-queues 16
        transaction-threads-per-queue 4
        proto-fd-max 15000
}

logging {
        # Log file must be an absolute path.
        file /var/log/aerospike/aerospike.log {
                context any info
        }
}

network {
        service {
                address any
                port 3000
        }

        heartbeat {
                mode mesh
                port 3002 # Heartbeat port for this node.

                # List one or more other nodes, one ip-address & port per line:
                mesh-seed-address-port <IP> 3002
                mesh-seed-address-port <IP> 3002
                mesh-seed-address-port <IP> 3002
                mesh-seed-address-port <IP> 3002
                mesh-seed-address-port <IP> 3002

                interval 250
                timeout 10
        }

        fabric {
                port 3001
        }

        info {
                port 3003
        }
}

namespace FC {
        replication-factor 2
        memory-size 30G
        storage-engine memory
                default-ttl 30d # 30 days, use 0 to never expire/evict.

        high-water-memory-pct 80 # Evict non-zero TTL data if capacity exceeds # 70% of 15GB
        stop-writes-pct 90       # Stop writes if capacity exceeds 90% of 15GB

}
</code>```

We began with the FC namespace only, and decided to go ahead with the HEAVYNAMESPACE only if we saw significant improvements with the FC namespace, but we didn't. Here are the current observations with different combinations of node count and server count -

Current stats

Observation Point 1 - 4 nodes serving 130 servers.
Point 2 - 5 nodes serving 80 servers.
Point 3 - 5 nodes serving 100 servers.

These observation points are highlighted in the graphs below -

Get latency -

Set successes (giving a measure of the load handled by the cluster) -

We also observed that -

Total memory usage across cluster is 5.52 GB of 144 GB. Node-wise memory usage is ~ 1.10 GB out of 28.90 GB.

There were no observed write failures yet.

There were occasional get/set timeouts which looked fine.

No evicted objects.

Conclusion

We are not seeing the improvements we had expected, by using the memory-only configuration. We would like to get some pointers to be able to scale up with the same cost -
 - via tweaking the aerospike configurations
 - or by using some more suitable AWS instance type (even if that would lead to cost cutting).

Update

Output of top command on one of the aerospike servers, to show SI (Pointed out by @Sunil in his answer) -

```<code>$ top
top - 08:02:21 up 188 days, 48 min,  1 user,  load average: 0.07, 0.07, 0.02
Tasks: 179 total,   1 running, 178 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.3%us,  0.1%sy,  0.0%ni, 99.4%id,  0.0%wa,  0.0%hi,  0.2%si,  0.0%st
Mem:  125904196k total,  2726964k used, 123177232k free,   148612k buffers
Swap:        0k total,        0k used,        0k free,   445968k cached

   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 63421 root      20   0 5217m 1.6g 4340 S  6.3  1.3 461:08.83 asd
</code>```

If I am not wrong, the SI appears to be 0.2%. I checked the same on all the nodes of the cluster and it is 0.2% on one and 0.1% on the rest of the three.

Also, here is the output of the network stats on the same node -

```<code>$ sar -n DEV 10 10
Linux 4.4.30-32.54.amzn1.x86_64 (ip-10-111-215-72)      07/10/17        _x86_64_        (16 CPU)

08:09:16        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:09:26           lo     12.20     12.20      5.61      5.61      0.00      0.00      0.00      0.00
08:09:26         eth0   2763.60   1471.60    299.24    233.08      0.00      0.00      0.00      0.00

08:09:26        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:09:36           lo     12.00     12.00      5.60      5.60      0.00      0.00      0.00      0.00
08:09:36         eth0   2772.60   1474.50    300.08    233.48      0.00      0.00      0.00      0.00

08:09:36        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:09:46           lo     17.90     17.90     15.21     15.21      0.00      0.00      0.00      0.00
08:09:46         eth0   2802.80   1491.90    304.63    245.33      0.00      0.00      0.00      0.00

08:09:46        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:09:56           lo     12.00     12.00      5.60      5.60      0.00      0.00      0.00      0.00
08:09:56         eth0   2805.20   1494.30    304.37    237.51      0.00      0.00      0.00      0.00

08:09:56        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:10:06           lo      9.40      9.40      5.05      5.05      0.00      0.00      0.00      0.00
08:10:06         eth0   3144.10   1702.30    342.54    255.34      0.00      0.00      0.00      0.00

08:10:06        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:10:16           lo     12.00     12.00      5.60      5.60      0.00      0.00      0.00      0.00
08:10:16         eth0   2862.70   1522.20    310.15    238.32      0.00      0.00      0.00      0.00

08:10:16        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:10:26           lo     12.00     12.00      5.60      5.60      0.00      0.00      0.00      0.00
08:10:26         eth0   2738.40   1453.80    295.85    231.47      0.00      0.00      0.00      0.00

08:10:26        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:10:36           lo     11.79     11.79      5.59      5.59      0.00      0.00      0.00      0.00
08:10:36         eth0   2758.14   1464.14    297.59    231.47      0.00      0.00      0.00      0.00

08:10:36        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:10:46           lo     12.00     12.00      5.60      5.60      0.00      0.00      0.00      0.00
08:10:46         eth0   3100.40   1811.30    328.31    289.92      0.00      0.00      0.00      0.00

08:10:46        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:10:56           lo      9.40      9.40      5.05      5.05      0.00      0.00      0.00      0.00
08:10:56         eth0   2753.40   1460.80    297.15    231.98      0.00      0.00      0.00      0.00

Average:        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
Average:           lo     12.07     12.07      6.45      6.45      0.00      0.00      0.00      0.00
Average:         eth0   2850.12   1534.68    307.99    242.79      0.00      0.00      0.00      0.00
</code>```

From the above, I think the total number of packets handled per second should be 2850.12+1534.68 = 4384.8 (sum of rxpck/s and txpck/s) which is well within 250K packets per second, as mentioned in The Amazon EC2 deployment guide on the Aerospike site which is referred in @RonenBotzer's answer.

Update 2

I ran the asadm command followed by show latency on one of the nodes of the cluster and from the output, it appears that there is no latency beyond 1 ms for both reads and writes -

```<code>Admin> show latency
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~read Latency~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                               Node                 Time   Ops/Sec   >1Ms   >8Ms   >64Ms
                                  .                 Span         .      .      .       .
ip-10-111-215-72.ec2.internal:3000    11:35:01->11:35:11    1242.1    0.0    0.0     0.0
ip-10-13-215-20.ec2.internal:3000     11:34:57->11:35:07    1297.5    0.0    0.0     0.0
ip-10-150-147-167.ec2.internal:3000   11:35:04->11:35:14    1147.7    0.0    0.0     0.0
ip-10-165-168-246.ec2.internal:3000   11:34:59->11:35:09    1342.2    0.0    0.0     0.0
ip-10-233-158-213.ec2.internal:3000   11:35:00->11:35:10    1218.0    0.0    0.0     0.0
Number of rows: 5

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~write Latency~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                               Node                 Time   Ops/Sec   >1Ms   >8Ms   >64Ms
                                  .                 Span         .      .      .       .
ip-10-111-215-72.ec2.internal:3000    11:35:01->11:35:11      33.0    0.0    0.0     0.0
ip-10-13-215-20.ec2.internal:3000     11:34:57->11:35:07      37.2    0.0    0.0     0.0
ip-10-150-147-167.ec2.internal:3000   11:35:04->11:35:14      36.4    0.0    0.0     0.0
ip-10-165-168-246.ec2.internal:3000   11:34:59->11:35:09      36.9    0.0    0.0     0.0
ip-10-233-158-213.ec2.internal:3000   11:35:00->11:35:10      33.9    0.0    0.0     0.0
Number of rows: 5
</code>```

答:
Your title is misleading. Please consider changing it. You moved from on-disk to in-memory.
mem+disk means data is both on disk and mem (using data-in-memory=true).

My best guess is that one CPU is bottlenecking to do network I/O.
You can take a look at the top output and see the si (software interrupts)
If one CPU is showing much higher than the other,
simplest thing you can try is RPS (Receive Packet Steering)

```<code>echo f|sudo tee  /sys/class/net/eth0/queues/rx-0/rps_cpus
</code>```

Once you confirm that its network bottlneck,
You can try ENA as suggested by @Ronen

Going into details,
When you had 15ms latency with only FC, assuming its low tps.
But when you added high load on HEAVYNAMESPACE in prod,
the latency kept increasing as you added more client nodes and hence tps.

Simlarly in you POC also, the latency increased with client nodes.
The latency is under 15ms even with 130 servers. Its partly good.
I am not sure if I understood your set_success graph. Assumign its in ktps.

Update:

After looking at the server side latency histogram, looks like server is doing fine.
Most likely it is a client issue. Check CPU and network on the client machine(s).

Body:
To begin with, we had an aerospike cluster having 5 nodes of i2.2xlarge type in AWS, which our production fleet of around 200 servers was using to store/retrieve data. The aerospike config of the cluster was as follows -

service {
        user root
        group root
        paxos-single-replica-limit 1 # Number of nodes where the replica count is automatically reduced to 1.
        pidfile /var/run/aerospike/asd.pid
        service-threads 8
        transaction-queues 8
        transaction-threads-per-queue 4
        fabric-workers 8
        transaction-pending-limit 100
        proto-fd-max 25000
}

logging {
        # Log file must be an absolute path.
        file /var/log/aerospike/aerospike.log {
                context any info
        }
}

network {
        service {
                address any
                port 3000
        }

        heartbeat {
                mode mesh
                port 3002 # Heartbeat port for this node.

                # List one or more other nodes, one ip-address & port per line:
                mesh-seed-address-port <IP> 3002
                mesh-seed-address-port <IP> 3002
                mesh-seed-address-port <IP> 3002
                mesh-seed-address-port <IP> 3002
               # mesh-seed-address-port <IP> 3002
                interval 250
                timeout 10
        }

        fabric {
                port 3001
        }

        info {
                port 3003
        }
}

namespace FC {
        replication-factor 2
        memory-size 7G
        default-ttl 30d # 30 days, use 0 to never expire/evict.

        high-water-disk-pct 80    # How full may the disk become before the server begins eviction
            high-water-memory-pct 70 # Evict non-zero TTL data if capacity exceeds # 70% of 15GB
               stop-writes-pct 90       # Stop writes if capacity exceeds 90% of 15GB

                  storage-engine device {
                  device /dev/xvdb1
                  write-block-size 256K
                      }

}

It was properly handling the traffic corresponding to the namespace "FC", with latencies within 14 ms, as shown in the following graph plotted using graphite -

However, on turning on another namespace, with much higher traffic on the same cluster, it started to give a lot of timeouts and higher latencies, as we scaled up the number of servers using the same cluster of 5 nodes (increasing number of servers step by step from 20 to 40 to 60) with the following namespace configuration -

namespace HEAVYNAMESPACE {
        replication-factor 2
        memory-size 35G
        default-ttl 30d # 30 days, use 0 to never expire/evict.

 high-water-disk-pct 80    # How full may the disk become before the server begins eviction
    high-water-memory-pct 70 # Evict non-zero TTL data if capacity exceeds # 70% of 35GB
   stop-writes-pct 90       # Stop writes if capacity exceeds 90% of 35GB

   storage-engine device {
device /dev/xvdb8
write-block-size 256K
    }

}

Following were the observations -

----FC Namespace----

20 - servers, 6k Write TPS, 16K Read TPS
set latency = 10ms
set timeouts = 1
get latency = 15ms
get timeouts = 3

40 - servers, 12k Write TPS, 17K Read TPS
set latency = 12ms
set timeouts = 1
get latency = 20ms
get timeouts = 5

60 - servers, 17k Write TPS, 18K Read TPS
set latency = 25ms
set timeouts = 5
get latency = 30ms
get timeouts = 10-50 (fluctuating)

----HEAVYNAMESPACE----

20 - del servers, 6k Write TPS, 16K Read TPS
set latency = 7ms
set timeouts = 1
get latency = 5ms
get timeouts = 0
no of keys = 47 million x 2
disk usage = 121 gb
ram usage = 5.62 gb

40 - del servers, 12k Write TPS, 17K Read TPS
set latency = 15ms
set timeouts = 5
get latency = 12ms
get timeouts = 2

60 - del servers, 17k Write TPS, 18K Read TPS
set latency = 25ms
set timeouts = 25-75 (fluctuating)
get latency = 25ms
get timeouts = 2-15 (fluctuating)

* Set latency refers to latency in setting aerospike cache keys and similarly get for getting keys.

We had to turn off the namespace "HEAVYNAMESPACE" after reaching 60 servers.

We then started a fresh POC with a cluster having nodes which were r3.4xlarge instances of AWS (find details here https://aws.amazon.com/ec2/instance-types/), with the key difference in aerospike configuration being the usage of memory only for caching, hoping that it would give better performance. Here is the aerospike.conf file -

service {
        user root
        group root
        paxos-single-replica-limit 1 # Number of nodes where the replica count is automatically reduced to 1.
        pidfile /var/run/aerospike/asd.pid
        service-threads 16
        transaction-queues 16
        transaction-threads-per-queue 4
        proto-fd-max 15000
}

logging {
        # Log file must be an absolute path.
        file /var/log/aerospike/aerospike.log {
                context any info
        }
}

network {
        service {
                address any
                port 3000
        }

        heartbeat {
                mode mesh
                port 3002 # Heartbeat port for this node.

                # List one or more other nodes, one ip-address & port per line:
                mesh-seed-address-port <IP> 3002
                mesh-seed-address-port <IP> 3002
                mesh-seed-address-port <IP> 3002
                mesh-seed-address-port <IP> 3002
                mesh-seed-address-port <IP> 3002

                interval 250
                timeout 10
        }

        fabric {
                port 3001
        }

        info {
                port 3003
        }
}

namespace FC {
        replication-factor 2
        memory-size 30G
        storage-engine memory
                default-ttl 30d # 30 days, use 0 to never expire/evict.

        high-water-memory-pct 80 # Evict non-zero TTL data if capacity exceeds # 70% of 15GB
        stop-writes-pct 90       # Stop writes if capacity exceeds 90% of 15GB

}

We began with the FC namespace only, and decided to go ahead with the HEAVYNAMESPACE only if we saw significant improvements with the FC namespace, but we didn't. Here are the current observations with different combinations of node count and server count -

Current stats

Observation Point 1 - 4 nodes serving 130 servers.
Point 2 - 5 nodes serving 80 servers.
Point 3 - 5 nodes serving 100 servers.

These observation points are highlighted in the graphs below -

Get latency -

Set successes (giving a measure of the load handled by the cluster) -

We also observed that -

Total memory usage across cluster is 5.52 GB of 144 GB. Node-wise memory usage is ~ 1.10 GB out of 28.90 GB.

There were no observed write failures yet.

There were occasional get/set timeouts which looked fine.

No evicted objects.

Conclusion

We are not seeing the improvements we had expected, by using the memory-only configuration. We would like to get some pointers to be able to scale up with the same cost -
 - via tweaking the aerospike configurations
 - or by using some more suitable AWS instance type (even if that would lead to cost cutting).

Update

Output of top command on one of the aerospike servers, to show SI (Pointed out by @Sunil in his answer) -

$ top
top - 08:02:21 up 188 days, 48 min,  1 user,  load average: 0.07, 0.07, 0.02
Tasks: 179 total,   1 running, 178 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.3%us,  0.1%sy,  0.0%ni, 99.4%id,  0.0%wa,  0.0%hi,  0.2%si,  0.0%st
Mem:  125904196k total,  2726964k used, 123177232k free,   148612k buffers
Swap:        0k total,        0k used,        0k free,   445968k cached

   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 63421 root      20   0 5217m 1.6g 4340 S  6.3  1.3 461:08.83 asd

If I am not wrong, the SI appears to be 0.2%. I checked the same on all the nodes of the cluster and it is 0.2% on one and 0.1% on the rest of the three.

Also, here is the output of the network stats on the same node -

$ sar -n DEV 10 10
Linux 4.4.30-32.54.amzn1.x86_64 (ip-10-111-215-72)      07/10/17        _x86_64_        (16 CPU)

08:09:16        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:09:26           lo     12.20     12.20      5.61      5.61      0.00      0.00      0.00      0.00
08:09:26         eth0   2763.60   1471.60    299.24    233.08      0.00      0.00      0.00      0.00

08:09:26        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:09:36           lo     12.00     12.00      5.60      5.60      0.00      0.00      0.00      0.00
08:09:36         eth0   2772.60   1474.50    300.08    233.48      0.00      0.00      0.00      0.00

08:09:36        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:09:46           lo     17.90     17.90     15.21     15.21      0.00      0.00      0.00      0.00
08:09:46         eth0   2802.80   1491.90    304.63    245.33      0.00      0.00      0.00      0.00

08:09:46        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:09:56           lo     12.00     12.00      5.60      5.60      0.00      0.00      0.00      0.00
08:09:56         eth0   2805.20   1494.30    304.37    237.51      0.00      0.00      0.00      0.00

08:09:56        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:10:06           lo      9.40      9.40      5.05      5.05      0.00      0.00      0.00      0.00
08:10:06         eth0   3144.10   1702.30    342.54    255.34      0.00      0.00      0.00      0.00

08:10:06        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:10:16           lo     12.00     12.00      5.60      5.60      0.00      0.00      0.00      0.00
08:10:16         eth0   2862.70   1522.20    310.15    238.32      0.00      0.00      0.00      0.00

08:10:16        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:10:26           lo     12.00     12.00      5.60      5.60      0.00      0.00      0.00      0.00
08:10:26         eth0   2738.40   1453.80    295.85    231.47      0.00      0.00      0.00      0.00

08:10:26        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:10:36           lo     11.79     11.79      5.59      5.59      0.00      0.00      0.00      0.00
08:10:36         eth0   2758.14   1464.14    297.59    231.47      0.00      0.00      0.00      0.00

08:10:36        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:10:46           lo     12.00     12.00      5.60      5.60      0.00      0.00      0.00      0.00
08:10:46         eth0   3100.40   1811.30    328.31    289.92      0.00      0.00      0.00      0.00

08:10:46        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
08:10:56           lo      9.40      9.40      5.05      5.05      0.00      0.00      0.00      0.00
08:10:56         eth0   2753.40   1460.80    297.15    231.98      0.00      0.00      0.00      0.00

Average:        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
Average:           lo     12.07     12.07      6.45      6.45      0.00      0.00      0.00      0.00
Average:         eth0   2850.12   1534.68    307.99    242.79      0.00      0.00      0.00      0.00

From the above, I think the total number of packets handled per second should be 2850.12+1534.68 = 4384.8 (sum of rxpck/s and txpck/s) which is well within 250K packets per second, as mentioned in The Amazon EC2 deployment guide on the Aerospike site which is referred in @RonenBotzer's answer.

Update 2

I ran the asadm command followed by show latency on one of the nodes of the cluster and from the output, it appears that there is no latency beyond 1 ms for both reads and writes -

Admin> show latency
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~read Latency~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                               Node                 Time   Ops/Sec   >1Ms   >8Ms   >64Ms
                                  .                 Span         .      .      .       .
ip-10-111-215-72.ec2.internal:3000    11:35:01->11:35:11    1242.1    0.0    0.0     0.0
ip-10-13-215-20.ec2.internal:3000     11:34:57->11:35:07    1297.5    0.0    0.0     0.0
ip-10-150-147-167.ec2.internal:3000   11:35:04->11:35:14    1147.7    0.0    0.0     0.0
ip-10-165-168-246.ec2.internal:3000   11:34:59->11:35:09    1342.2    0.0    0.0     0.0
ip-10-233-158-213.ec2.internal:3000   11:35:00->11:35:10    1218.0    0.0    0.0     0.0
Number of rows: 5

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~write Latency~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                               Node                 Time   Ops/Sec   >1Ms   >8Ms   >64Ms
                                  .                 Span         .      .      .       .
ip-10-111-215-72.ec2.internal:3000    11:35:01->11:35:11      33.0    0.0    0.0     0.0
ip-10-13-215-20.ec2.internal:3000     11:34:57->11:35:07      37.2    0.0    0.0     0.0
ip-10-150-147-167.ec2.internal:3000   11:35:04->11:35:14      36.4    0.0    0.0     0.0
ip-10-165-168-246.ec2.internal:3000   11:34:59->11:35:09      36.9    0.0    0.0     0.0
ip-10-233-158-213.ec2.internal:3000   11:35:00->11:35:10      33.9    0.0    0.0     0.0
Number of rows: 5

Body:
Your title is misleading. Please consider changing it. You moved from on-disk to in-memory.
mem+disk means data is both on disk and mem (using data-in-memory=true).

My best guess is that one CPU is bottlenecking to do network I/O.
You can take a look at the top output and see the si (software interrupts)
If one CPU is showing much higher than the other,
simplest thing you can try is RPS (Receive Packet Steering)

echo f|sudo tee  /sys/class/net/eth0/queues/rx-0/rps_cpus

Once you confirm that its network bottlneck,
You can try ENA as suggested by @Ronen

Going into details,
When you had 15ms latency with only FC, assuming its low tps.
But when you added high load on HEAVYNAMESPACE in prod,
the latency kept increasing as you added more client nodes and hence tps.

Simlarly in you POC also, the latency increased with client nodes.
The latency is under 15ms even with 130 servers. Its partly good.
I am not sure if I understood your set_success graph. Assumign its in ktps.

Update:

After looking at the server side latency histogram, looks like server is doing fine.
Most likely it is a client issue. Check CPU and network on the client machine(s).


来源: stackexchange

问:
What's the usage of a a def in a function?
Just reading the Python doc's in particular this page, in regards to attrgetter and itemgetter.

I'm struggling to understand the usage of the usage of def and return in the code examples. Not the usage of those functions.

It seems like a lamdba, but I don't know how to interpret it.

EDIT

I think it just clicked. The attrgetter is like a constructor for the resolve_attr function? This was a little confusing, as i'd normally expect the code for a def to be between the def and the return. The code on the return is like a lambda. To add to the confusion the arg ojb is the argument passed to the returned object. Arhhhhh.....

What do i need to know for this to make sense?

This is the code example I'm referring to:

```<code>   def attrgetter(*items):
    if any(not isinstance(item, str) for item in items):
        raise TypeError('attribute name must be a string')
    if len(items) == 1:
        attr = items[0]
        def g(obj): #What is this
            return resolve_attr(obj, attr)
    else:
        def g(obj):#And this?
            return tuple(resolve_attr(obj, attr) for attr in items)
    return g

def resolve_attr(obj, attr):
    for name in attr.split("."):
        obj = getattr(obj, name)
    return obj
</code>```

答:
def is the keyword to define a function.

These are just functions defined inside another function  attrgetter()

That's why you have multiple return inside attrgetter()

g looks like lambda because attrgetter()  returns a function, and not the result directly.

g could be replaced by a lambda declaration indeed.

ContentLicense: CC BY-SA 3.0

Body:
Just reading the Python doc's in particular this page, in regards to attrgetter and itemgetter.

I'm struggling to understand the usage of the usage of def and return in the code examples. Not the usage of those functions.

It seems like a lamdba, but I don't know how to interpret it.

EDIT

I think it just clicked. The attrgetter is like a constructor for the resolve_attr function? This was a little confusing, as i'd normally expect the code for a def to be between the def and the return. The code on the return is like a lambda. To add to the confusion the arg ojb is the argument passed to the returned object. Arhhhhh.....

What do i need to know for this to make sense?

This is the code example I'm referring to:

   def attrgetter(*items):
    if any(not isinstance(item, str) for item in items):
        raise TypeError('attribute name must be a string')
    if len(items) == 1:
        attr = items[0]
        def g(obj): #What is this
            return resolve_attr(obj, attr)
    else:
        def g(obj):#And this?
            return tuple(resolve_attr(obj, attr) for attr in items)
    return g

def resolve_attr(obj, attr):
    for name in attr.split("."):
        obj = getattr(obj, name)
    return obj


问:
What are GC roots for classes?
In Java, there are special objects called Garbage Collection Roots (GC roots). They serve as a root objects for Garbage Collection marking mechanism (see picture).

This article describes four types of GC roots:

local variables

active threads

static variables

JNI references

It is also mentioned, that:

Classes themselves can be garbage-collected.

GC roots aren't collected thus classes themselves are not GC roots.

So what are GC roots for the classes?

答:
A garbage collection root is an object that is accessible from outside the heap.

Memory Analyzer categorizes garbage collection roots according to the following list:

Class loaded by system ClassLoader

static field in JDK classes(java.* etc)

Live thread

stack -local vars, method params

java.lang.Thread instance

Object held as synchronization monitor

JNI references

JVM specials...

Source 1
Source 2

Title: What are GC roots for classes?

ContentLicense: CC BY-SA 4.0


问:
How to update Php web page when db data changes from another device with android app?
Which is the correct way to do this: I have a pc with a php web application (as admin user) to "control" data (for example notes) but the notes are inserted in the db through a web service(APiRest) from android devices(using a simple app) which other users inserts in real time, but these notes must be checked by the admin in the pc(with a boolean value, e.g. "checked" or not). So, how to automatically update the admin web page when a user from the android app inserts a new note? Should I use triggers? What is the correct technique? Any suggestions are welcome, thanks in advance.</p>

答:
I would add a column to your table (for example approved), and set its default value to 0. When the user adds a note from the app (adds a row to the table), the approved column will be set to 0. In the PHP admin panel, you can modify the select query like so:

SELECT * FROM table_name WHERE approved=0

With this, only the rows that have not been approved will be shown, allowing the admin to approve them.


问:
Neural network gives different output for same input
What are the potential reasons for a NN to output different values for the same input? Especially when there isn't any random or stochastic processes?</p>

答: It might be due to shuffling of data , If you want to use the same vector you should turn the shuffle argument off.</p>


问:
Java: Volatile variable not updating (get and set methods not working)
I have a Runnable "NanoClock" class which keeps updating a private volatile double value in its run() method.

This class also has a getTime() method which returns the double value. Another class ("Master") is constructing the NanoClock class and creates a thread, as well as calling the start() method.

After it did this it calls the getTime() method several times (with a delay), but the value is not updating. What am I doing wrong?

NanoClock.java :

```<code>public class NanoClock implements Runnable {
    private volatile boolean running;
    private volatile double time;
    public NanoClock() {
        time = System.currentTimeMillis();
    }
    @Override
    public void run() {
        running = true;
        while(running) {
            try {
                if(System.currentTimeMillis() > time) {
                    time = System.currentTimeMillis();
                }
              //This returns the updated value continuously when commented out
              //System.out.println("Time: " + String.format("%.6f", unix_time));
                Thread.sleep(2000);
            } catch(Exception exc) {
                exc.printStackTrace();
                System.exit(1);
            }
        }
    }
    public double getTime() {
        return time;
    }
    public void end() {
        running = false;
    }
}
</code>```

Master.java:

```<code>public class Master {
    public static void main(String[] args) {
        try {
            NanoClock nClock = new NanoClock();
            Thread clockThread = new Thread(new NanoClock());
            clockThread.setPriority(10);
            clockThread.start();
            //MY_ISSUE: This returns the same value every time
            for(int a = 0; a < 10; a++) {
                System.out.println("Time: " + nClock.getTime());
            }
            //MY_ISSUE: This cannot stop the while loop - I tested it with
            //the println in the NanoClock class.
            nClock.end();
            System.out.println("Done!");
        catch(Exception e) {
            e.printStackTrace();
            System.exit(1);
        }
    }
}
</code>```

答:
```<code>Thread.sleep(2000);
System.out.println("Time: " + nClock.getTime());
</code>```

the for in main() must be sleep(2000)

Body:
I have a Runnable "NanoClock" class which keeps updating a private volatile double value in its run() method.

This class also has a getTime() method which returns the double value. Another class ("Master") is constructing the NanoClock class and creates a thread, as well as calling the start() method.

After it did this it calls the getTime() method several times (with a delay), but the value is not updating. What am I doing wrong?

NanoClock.java :

public class NanoClock implements Runnable {
    private volatile boolean running;
    private volatile double time;
    public NanoClock() {
        time = System.currentTimeMillis();
    }
    @Override
    public void run() {
        running = true;
        while(running) {
            try {
                if(System.currentTimeMillis() > time) {
                    time = System.currentTimeMillis();
                }
              //This returns the updated value continuously when commented out
              //System.out.println("Time: " + String.format("%.6f", unix_time));
                Thread.sleep(2000);
            } catch(Exception exc) {
                exc.printStackTrace();
                System.exit(1);
            }
        }
    }
    public double getTime() {
        return time;
    }
    public void end() {
        running = false;
    }
}

Master.java:

public class Master {
    public static void main(String[] args) {
        try {
            NanoClock nClock = new NanoClock();
            Thread clockThread = new Thread(new NanoClock());
            clockThread.setPriority(10);
            clockThread.start();
            //MY_ISSUE: This returns the same value every time
            for(int a = 0; a < 10; a++) {
                System.out.println("Time: " + nClock.getTime());
            }
            //MY_ISSUE: This cannot stop the while loop - I tested it with
            //the println in the NanoClock class.
            nClock.end();
            System.out.println("Done!");
        catch(Exception e) {
            e.printStackTrace();
            System.exit(1);
        }
    }
}

Body:
Thread.sleep(2000);
System.out.println("Time: " + nClock.getTime());

the for in main() must be sleep(2000)


问:
Putty gives "Network Error: Connection time out" while connecting to raspberry pi 3 using router in windows
I am using D'link rouer DSL-2750U.  I have installed raspbian jessie in Rpi 3 and connected it to router. Connected my laptop to router wifi. But when i try to connect putty using the IP address of raspberrypi which i have got from router admin panel it gives connection time out error. Also i have tried to adjust keepalive seconds in putty connection setting and enabled X11 forwarding.</p>

答: Please Open you RaspberryPI terminal and run the command "sudo service ssh restart". Try connect again from putty</p>


问:
Hibernate arrays?
i have a POJO and a hibernate that holds integers that specify the points system for a particular race(i.e 1st place = 100, 2nd place = 70......) and was wondering if their is a way to create an array object and map it in my hbm file and pojo class.

here is how it looks at the moment:

```<code><?xml version="1.0"?>
<!DOCTYPE hibernate-mapping PUBLIC "-//Hibernate/Hibernate Mapping DTD 3.0//EN"
                                   "http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd">

<hibernate-mapping>
    <class name="com.jr.model.RaceSeriesPointsRules" table="RACE_SERIES_POINTS_RULES">
        <id name="id" type="string">
            <column name="ID" />
            <generator class="uuid" />
        </id>

        <property name="pos1" type="integer" column="POS1" />
        <property name="pos2" type="integer" column="POS2" />
        <property name="pos3" type="integer" column="POS3" />
        <property name="pos4" type="integer" column="POS54" />
        <property name="pos5" type="integer" column="POS5" />

        <property name="pos6" type="integer" column="POS6" />
        <property name="pos7" type="integer" column="POS7" />
        <property name="pos8" type="integer" column="POS8" />
        <property name="pos9" type="integer" column="POS9" />

        <property name="pos10" type="integer" column="POS10" />
        <property name="pos11" type="integer" column="POS11" />
        <property name="pos12" type="integer" column="POS12" />
        <property name="pos13" type="integer" column="POS13" />

        <property name="pos14" type="integer" column="POS14" />
        <property name="pos15" type="integer" column="POS15" />
        <property name="pos16" type="integer" column="POS16" />

    </class>

</hibernate-mapping>
</code>```

Here is my pojo below:

```<code>package com.jr.model;

public class RaceSeriesPointsRules {

    private String id;

    private int pos1;
    private int pos2;
    private int pos3;
    private int pos4;
    private int pos5;
    private int pos6;
    private int pos7;
    private int pos8;
    private int pos9;
    private int pos10;
    private int pos11;
    private int pos12;
    private int pos13;
    private int pos14;
    private int pos15;
    private int pos16;

    public RaceSeriesPointsRules(int pos1, int pos2, int pos3, int pos4,
            int pos5, int pos6, int pos7, int pos8, int pos9, int pos10,
            int pos11, int pos12, int pos13, int pos14, int pos15, int pos16) {

        this.pos1 = pos1;
        this.pos2 = pos2;
        this.pos3 = pos3;
        this.pos4 = pos4;
        this.pos5 = pos5;
        this.pos6 = pos6;
        this.pos7 = pos7;
        this.pos8 = pos8;
        this.pos9 = pos9;
        this.pos10 = pos10;
        this.pos11 = pos11;
        this.pos12 = pos12;
        this.pos13 = pos13;
        this.pos14 = pos14;
        this.pos15 = pos15;
        this.pos16 = pos16;
    }

    public RaceSeriesPointsRules(){

    }

    public String getId() {
        return id;
    }
    public void setId(String id) {
        this.id = id;
    }
    public int getPos1() {
        return pos1;
    }
    public void setPos1(int pos1) {
        this.pos1 = pos1;
    }
    public int getPos2() {
        return pos2;
    }
    public void setPos2(int pos2) {
        this.pos2 = pos2;
    }
    public int getPos3() {
        return pos3;
    }
    public void setPos3(int pos3) {
        this.pos3 = pos3;
    }
    public int getPos4() {
        return pos4;
    }
    public void setPos4(int pos4) {
        this.pos4 = pos4;
    }
    public int getPos5() {
        return pos5;
    }
    public void setPos5(int pos5) {
        this.pos5 = pos5;
    }
    public int getPos6() {
        return pos6;
    }
    public void setPos6(int pos6) {
        this.pos6 = pos6;
    }
    public int getPos7() {
        return pos7;
    }
    public void setPos7(int pos7) {
        this.pos7 = pos7;
    }
    public int getPos8() {
        return pos8;
    }
    public void setPos8(int pos8) {
        this.pos8 = pos8;
    }
    public int getPos9() {
        return pos9;
    }
    public void setPos9(int pos9) {
        this.pos9 = pos9;
    }
    public int getPos10() {
        return pos10;
    }
    public void setPos10(int pos10) {
        this.pos10 = pos10;
    }
    public int getPos11() {
        return pos11;
    }
    public void setPos11(int pos11) {
        this.pos11 = pos11;
    }
    public int getPos12() {
        return pos12;
    }
    public void setPos12(int pos12) {
        this.pos12 = pos12;
    }
    public int getPos13() {
        return pos13;
    }
    public void setPos13(int pos13) {
        this.pos13 = pos13;
    }
    public int getPos14() {
        return pos14;
    }
    public void setPos14(int pos14) {
        this.pos14 = pos14;
    }
    public int getPos15() {
        return pos15;
    }
    public void setPos15(int pos15) {
        this.pos15 = pos15;
    }
    public int getPos16() {
        return pos16;
    }
    public void setPos16(int pos16) {
        this.pos16 = pos16;
    }

}
</code>```

As you can see its a hack job and would like to know if i can use arrays in hibernate to create columns of array.size and populate them with ints. this will make my pojo very small and do logic in calculating the pts that a racer has accumulated based on the position they finished:

racer A finishes 2nd, racer a position = 2;

Select column.pos2 from race_series_points_rules where .........

edit: another question is what happens if race series A has only 10 positions and Race Series B has 16? can both raceSeriesPointsRules be stores in the same table even though both rows will have different column count?

Hope my explanations are clear. Thanks in advance.

答:
I would use an indexed collection for that, maybe even doing your own implementation of List (delegating all methods to the actual implementation, but allowing only 10 elements). This way, you make sure to have at most 10 items, and you'll still keep their positions. You can also easily calculate the points by looking at the index.

http://docs.jboss.org/hibernate/core/3.3/reference/en/html/collections.html#collections-indexed

ContentLicense: CC BY-SA 2.5

Title: Hibernate arrays?

Body:
i have a POJO and a hibernate that holds integers that specify the points system for a particular race(i.e 1st place = 100, 2nd place = 70......) and was wondering if their is a way to create an array object and map it in my hbm file and pojo class.

here is how it looks at the moment:

<?xml version="1.0"?>
<!DOCTYPE hibernate-mapping PUBLIC "-//Hibernate/Hibernate Mapping DTD 3.0//EN"
                                   "http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd">

<hibernate-mapping>
    <class name="com.jr.model.RaceSeriesPointsRules" table="RACE_SERIES_POINTS_RULES">
        <id name="id" type="string">
            <column name="ID" />
            <generator class="uuid" />
        </id>

        <property name="pos1" type="integer" column="POS1" />
        <property name="pos2" type="integer" column="POS2" />
        <property name="pos3" type="integer" column="POS3" />
        <property name="pos4" type="integer" column="POS54" />
        <property name="pos5" type="integer" column="POS5" />

        <property name="pos6" type="integer" column="POS6" />
        <property name="pos7" type="integer" column="POS7" />
        <property name="pos8" type="integer" column="POS8" />
        <property name="pos9" type="integer" column="POS9" />

        <property name="pos10" type="integer" column="POS10" />
        <property name="pos11" type="integer" column="POS11" />
        <property name="pos12" type="integer" column="POS12" />
        <property name="pos13" type="integer" column="POS13" />

        <property name="pos14" type="integer" column="POS14" />
        <property name="pos15" type="integer" column="POS15" />
        <property name="pos16" type="integer" column="POS16" />

    </class>

</hibernate-mapping>

Here is my pojo below:

package com.jr.model;

public class RaceSeriesPointsRules {

    private String id;

    private int pos1;
    private int pos2;
    private int pos3;
    private int pos4;
    private int pos5;
    private int pos6;
    private int pos7;
    private int pos8;
    private int pos9;
    private int pos10;
    private int pos11;
    private int pos12;
    private int pos13;
    private int pos14;
    private int pos15;
    private int pos16;

    public RaceSeriesPointsRules(int pos1, int pos2, int pos3, int pos4,
            int pos5, int pos6, int pos7, int pos8, int pos9, int pos10,
            int pos11, int pos12, int pos13, int pos14, int pos15, int pos16) {

        this.pos1 = pos1;
        this.pos2 = pos2;
        this.pos3 = pos3;
        this.pos4 = pos4;
        this.pos5 = pos5;
        this.pos6 = pos6;
        this.pos7 = pos7;
        this.pos8 = pos8;
        this.pos9 = pos9;
        this.pos10 = pos10;
        this.pos11 = pos11;
        this.pos12 = pos12;
        this.pos13 = pos13;
        this.pos14 = pos14;
        this.pos15 = pos15;
        this.pos16 = pos16;
    }

    public RaceSeriesPointsRules(){

    }

    public String getId() {
        return id;
    }
    public void setId(String id) {
        this.id = id;
    }
    public int getPos1() {
        return pos1;
    }
    public void setPos1(int pos1) {
        this.pos1 = pos1;
    }
    public int getPos2() {
        return pos2;
    }
    public void setPos2(int pos2) {
        this.pos2 = pos2;
    }
    public int getPos3() {
        return pos3;
    }
    public void setPos3(int pos3) {
        this.pos3 = pos3;
    }
    public int getPos4() {
        return pos4;
    }
    public void setPos4(int pos4) {
        this.pos4 = pos4;
    }
    public int getPos5() {
        return pos5;
    }
    public void setPos5(int pos5) {
        this.pos5 = pos5;
    }
    public int getPos6() {
        return pos6;
    }
    public void setPos6(int pos6) {
        this.pos6 = pos6;
    }
    public int getPos7() {
        return pos7;
    }
    public void setPos7(int pos7) {
        this.pos7 = pos7;
    }
    public int getPos8() {
        return pos8;
    }
    public void setPos8(int pos8) {
        this.pos8 = pos8;
    }
    public int getPos9() {
        return pos9;
    }
    public void setPos9(int pos9) {
        this.pos9 = pos9;
    }
    public int getPos10() {
        return pos10;
    }
    public void setPos10(int pos10) {
        this.pos10 = pos10;
    }
    public int getPos11() {
        return pos11;
    }
    public void setPos11(int pos11) {
        this.pos11 = pos11;
    }
    public int getPos12() {
        return pos12;
    }
    public void setPos12(int pos12) {
        this.pos12 = pos12;
    }
    public int getPos13() {
        return pos13;
    }
    public void setPos13(int pos13) {
        this.pos13 = pos13;
    }
    public int getPos14() {
        return pos14;
    }
    public void setPos14(int pos14) {
        this.pos14 = pos14;
    }
    public int getPos15() {
        return pos15;
    }
    public void setPos15(int pos15) {
        this.pos15 = pos15;
    }
    public int getPos16() {
        return pos16;
    }
    public void setPos16(int pos16) {
        this.pos16 = pos16;
    }

}

As you can see its a hack job and would like to know if i can use arrays in hibernate to create columns of array.size and populate them with ints. this will make my pojo very small and do logic in calculating the pts that a racer has accumulated based on the position they finished:

racer A finishes 2nd, racer a position = 2;

Select column.pos2 from race_series_points_rules where .........

edit: another question is what happens if race series A has only 10 positions and Race Series B has 16? can both raceSeriesPointsRules be stores in the same table even though both rows will have different column count?

Hope my explanations are clear. Thanks in advance.


问:
Parallel eternal loop (does not exit loop)
I'm currently using parallel to run a command in a defined amount of times as follow

```<code>   parallel -n0 -j2 "some command" ::: {1..999}
</code>```

Where:

-n0 = don't use input (in this case, from 1 till 999)

-j2 = 2 parallel instances of "some command"

Problem:
After reach 999, parallel command do not exit as expected and "some command" is not called anymore under supervisor.

Is there a way to parallel exit naturally after finish loop?

parallel --version
GNU parallel 20130922
Copyright (C) 2007,2008,2009,2010,2011,2012,2013 Ole Tange and Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
GNU parallel comes with no warranty.
Web site: http://www.gnu.org/software/parallel
When using GNU Parallel for a publication please cite:
O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
;login: The USENIX Magazine, February 2011:42-47.

答:
Does this work:

```<code>parallel -n0 -j2 echo ::: {1..999}
</code>```

If yes: The problem is in some command and you need to give us more information.

Body:
I'm currently using parallel to run a command in a defined amount of times as follow

   parallel -n0 -j2 "some command" ::: {1..999}

Where:

-n0 = don't use input (in this case, from 1 till 999)

-j2 = 2 parallel instances of "some command"

Problem:
After reach 999, parallel command do not exit as expected and "some command" is not called anymore under supervisor.

Is there a way to parallel exit naturally after finish loop?

parallel --version
GNU parallel 20130922
Copyright (C) 2007,2008,2009,2010,2011,2012,2013 Ole Tange and Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
GNU parallel comes with no warranty.
Web site: http://www.gnu.org/software/parallel
When using GNU Parallel for a publication please cite:
O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
;login: The USENIX Magazine, February 2011:42-47.

Body:
Does this work:

parallel -n0 -j2 echo ::: {1..999}

If yes: The problem is in some command and you need to give us more information.


问:
Better way to Check for values
Is there a more elegant way to do this?

```<code>if(response && response.body && response.body.data) {
  const users = response.body.data
  dispatch(UserActions.usersReceived(users))
}
</code>```

Meaning the response && response.body && response.body.data ?

答: No. But if it is a REST API (which looks like it is) I would recommend using a library for validations. And joi is the best when you have to perform nested validations.

Title: Better way to Check for values

Body:
Is there a more elegant way to do this?

if(response && response.body && response.body.data) {
  const users = response.body.data
  dispatch(UserActions.usersReceived(users))
}

Meaning the response && response.body && response.body.data ?


问:
Apart from maps and vectors are there any other uses of get in Clojure?
The uses that I know so far are on vectors:

```<code>(get [1 2 3 4] 2) ; => 3
</code>```

and on maps:

```<code>(get {:a "a" :b "B" :c "c"} :c) ; => "c"
</code>```

From documentation it says:

clojure.core/get ([map key] [map key not-found])

Returns the value mapped to key, not-found or nil if key not present.

答:
Further to @ToniVanhanla's answer, for the JVM, the relevant Clojure interface is clojure.lang.ILookup.

Looking, as the Americans say, under the hood,

Clojure get translates into a call to clojure.lang.RT/get.

If possible, this casts to ILookup and calls the appropriate
valAt method.

If not, it calls ...RT/getFrom

... whose clauses deal explicitly with, in turn,

Java maps,

Clojure sets, and

Java strings and arrays.

If none of these fits, it returns nil.

There is no parent interface for Java arrays: they all descend directly from Object. They are detected by Java's Class/isArray

Somewhat surprisingly, Clojure get does not work on Java collections such as Vectors:

```<code>(java.util.Vector. (seq "Hello, world!"))
=> [\H \e \l \l \o \, \space \w \o \r \l \d \!]
</code>```

But

```<code>(get (java.util.Vector. (seq "Hello, world!")) 4)
=> nil
</code>```

Body:
The uses that I know so far are on vectors:

(get [1 2 3 4] 2) ; => 3

and on maps:

(get {:a "a" :b "B" :c "c"} :c) ; => "c"

From documentation it says:

clojure.core/get ([map key] [map key not-found])

Returns the value mapped to key, not-found or nil if key not present.

Body:
Further to @ToniVanhanla's answer, for the JVM, the relevant Clojure interface is clojure.lang.ILookup.

Looking, as the Americans say, under the hood,

Clojure get translates into a call to clojure.lang.RT/get.

If possible, this casts to ILookup and calls the appropriate
valAt method.

If not, it calls ...RT/getFrom

... whose clauses deal explicitly with, in turn,

Java maps,

Clojure sets, and

Java strings and arrays.

If none of these fits, it returns nil.

There is no parent interface for Java arrays: they all descend directly from Object. They are detected by Java's Class/isArray

Somewhat surprisingly, Clojure get does not work on Java collections such as Vectors:

(java.util.Vector. (seq "Hello, world!"))
=> [\H \e \l \l \o \, \space \w \o \r \l \d \!]

But

(get (java.util.Vector. (seq "Hello, world!")) 4)
=> nil


问:
Start Ruby on Rails app through systemd socket activation (i.e. start Rails server listening on pre-existing socket)
As the title states, I'm trying to start a Ruby on Rails app through systemd socket activation. I've no prior experience with Ruby and the lack of quality documentation is frustrating.

Obtaining the listen socket from systemd is trivial enough, though I'm struggling to put that socket to use. Is there an app server, that accepts a pre-existing listen socket? Or do I have to write my own, possibly by implementing a Rack::Handler (going by what limited information I've been able to gather from the documentation)? In the latter case a minimal example implementation would be helpful.

I'm looking for a production-suitable, lightweight, least-effort solution, capable of serving an average of 2 concurrent users on embedded hardware.

答:
Starting with Puma 3.0.0, socket activation is supported by its Rack module, which is used by rails server. This means that running bin/rails server puma from systemd with socket activation will now work out of the box (tested with Puma 3.9.1). One caveat is that you must start bin/rails inside your application root, not the globally installed version.

For more details on this setup and some examples, see my blogpost.


问:
Css div selector multiple classes
I am working on a wordpress theme and I have a this code in HTML:

```<code><body>
        <div class="container-fluid mytheme-container" >
        <div class="container-fluid col-lg-offset-5 col-lg-7 404page">
                    <p id="404">404</p>
                    <p id="page-not-found">Page not found... :(</p>
        </div>
        </div>
</body>
</code>```

I want to add height to the last div and I thought my css in this way:

```<code>body > div.container-fluid.mytheme-container > div.container-fluid.col-lg-offset-5.col-lg-7.404page {
    height: 100vmin;
}
</code>```

That doesn't work, just if I let the selector like this:

```<code>body > div.container-fluid.mytheme-container > div {

}
</code>```

But I think that this will apply to every child of mytheme-container  and I don't want that. So, how can I get the selector for that div ( 404page ) in a more specific way than the last one?

答:
You want apply css to last div, flow this code.</p>

```<code>body > div.container-fluid.mytheme-container > div:last-child {

}
</code>```

Title: Css div selector multiple classes

Body:
I am working on a wordpress theme and I have a this code in HTML:

404

Page not found... :(

I want to add height to the last div and I thought my css in this way:

body > div.container-fluid.mytheme-container > div.container-fluid.col-lg-offset-5.col-lg-7.404page {
    height: 100vmin;
}

That doesn't work, just if I let the selector like this:

body > div.container-fluid.mytheme-container > div {

}

But I think that this will apply to every child of mytheme-container  and I don't want that. So, how can I get the selector for that div ( 404page ) in a more specific way than the last one?

Body:
You want apply css to last div, flow this code.

body > div.container-fluid.mytheme-container > div:last-child {

}


问:
Is it possible to require the C++ `override` specifier for virtual functions?
Is there any current way, or possibly planned standards change, that would allow me to require an overridden virtual function use the override keyword?

```<code>class base {
    virtual void foo() = 0; //what, if anything, can I do here so that...
}

class derived {
    void foo(); //...this is an error...
    void foo() override; //...and only this accepted
}
</code>```

答: No, there is no such feature in C++.  You could enforce it with a static analysis tool, perhaps.</p>

Body:
Is there any current way, or possibly planned standards change, that would allow me to require an overridden virtual function use the override keyword?

class base {
    virtual void foo() = 0; //what, if anything, can I do here so that...
}

class derived {
    void foo(); //...this is an error...
    void foo() override; //...and only this accepted
}


问:
Passport-Facebook authentication is not providing email or other information(undefined) for all Facebook accounts
I have tried everything but I am not able to get any other info. from fb account. I am only getting displayName and id from my facebook strategy :

```<code>passport.use(new FacebookStrategy({

    clientID        : configAuth.facebookAuth.clientID,
    clientSecret    : configAuth.facebookAuth.clientSecret,
    callbackURL     : configAuth.facebookAuth.callbackURL,
   // profileFeilds   : ['id','displayName','email'],
    passReqToCallback : true,
    profileFeilds   : ['id', 'email', 'gender', 'link', 'locale', 'name', 'timezone', 'updated_time', 'verified'],
 },
function( req,accessToken, refreshToken, profile, done) {

    //console.log(profile);
    console.log(JSON.stringify(profile));
    done(null,profile);

}));
</code>```

The get part:

```<code>    router.get('/auth/facebook', passport.authenticate('facebook', { scope : 'email' }));
    router.get('/auth/facebook/callback',
        passport.authenticate('facebook', {
        successRedirect : '/',
        failureRedirect : '/blogs',
        //scope:['email']
        }));
</code>```

Output:

```<code> { id: '1233755832057029',
username: undefined,
displayName: 'John Maverick',
name:
 { familyName: undefined,
  givenName: undefined,
  middleName: undefined },
gender: undefined,
profileUrl: undefined,
provider: 'facebook',
_raw: '{"name":"John Maverick","id":"1223755832057029"}',
_json: { name: 'John Maverick', id: '1223755832057029' } }
</code>```

答:
Change this

```<code>router.get('/auth/facebook', passport.authenticate('facebook', { scope : 'email' }));
</code>```

To this

```<code>router.get('/auth/facebook', passport.authenticate('facebook', { scope : ['email'] }));
</code>```

And change this:

```<code>profileFeilds
</code>```

To this:

```<code>profileFields
</code>```

Body:
I have tried everything but I am not able to get any other info. from fb account. I am only getting displayName and id from my facebook strategy :

passport.use(new FacebookStrategy({

    clientID        : configAuth.facebookAuth.clientID,
    clientSecret    : configAuth.facebookAuth.clientSecret,
    callbackURL     : configAuth.facebookAuth.callbackURL,
   // profileFeilds   : ['id','displayName','email'],
    passReqToCallback : true,
    profileFeilds   : ['id', 'email', 'gender', 'link', 'locale', 'name', 'timezone', 'updated_time', 'verified'],
 },
function( req,accessToken, refreshToken, profile, done) {

    //console.log(profile);
    console.log(JSON.stringify(profile));
    done(null,profile);

}));

The get part:

    router.get('/auth/facebook', passport.authenticate('facebook', { scope : 'email' }));
    router.get('/auth/facebook/callback',
        passport.authenticate('facebook', {
        successRedirect : '/',
        failureRedirect : '/blogs',
        //scope:['email']
        }));

Output:

 { id: '1233755832057029',
username: undefined,
displayName: 'John Maverick',
name:
 { familyName: undefined,
  givenName: undefined,
  middleName: undefined },
gender: undefined,
profileUrl: undefined,
provider: 'facebook',
_raw: '{"name":"John Maverick","id":"1223755832057029"}',
_json: { name: 'John Maverick', id: '1223755832057029' } }

Body:
Change this

router.get('/auth/facebook', passport.authenticate('facebook', { scope : 'email' }));

To this

router.get('/auth/facebook', passport.authenticate('facebook', { scope : ['email'] }));

And change this:

profileFeilds

To this:

profileFields


问:
Can I 'drop' a PHP request without replying?
I am not completely sure that this is possible from within PHP or even the HTTP protocol in depth. Is it possible to receive PHP script to run when it is requested but then the script drops the connection? Without replying?

Hopefully my question makes sense.

(I am using Apache)

答: If you don't want to send any data at all, use a sleep in a loop. Eventually the client end will time out and close the connection.


问:
How do I filter a stream of integers into a list?
I am trying to process a stream of Integers and collect the integers that match a predicate (via the compare() function) into a list. Here's a rough outline of the code I've written.

```<code>private List<Integer> process() {
    Z z = f(-1);
    return IntStream.range(0, 10)
        .filter(i -> compare(z, f(i)))
        .collect(Collectors.toCollection(ArrayList::new)); // Error on this line
}

private boolean compare(Z z1, Z z2) { ... }
private Z f(int i) { ... }
</code>```

Unfortunately my solution does not compile and I cannot make sense of the compiler error for the line highlighted:

The method collect(Supplier<R>, ObjIntConsumer<R>, BiConsumer<R,R>) in the type IntStream is not applicable for the arguments (Collector<Object,capture#1-of ?,Collection<Object>>)

Any suggestions?

答:
IntStream doesn't contain a collect method that accepts a single argument of type Collector. Stream does. Therefore you have to convert your IntStream to a Stream of objects.

You can either box the IntStream into a Stream<Integer> or use mapToObj to achieve the same.

For example:

```<code>return IntStream.range(0, 10)
    .filter(i -> compare(z, f(i)))
    .boxed()
    .collect(Collectors.toCollection(ArrayList::new));
</code>```

boxed() will return a Stream consisting of the elements of this stream, each boxed to an Integer.

or

```<code>return IntStream.range(0, 10)
    .filter(i -> compare(z, f(i)))
    .mapToObj(Integer::valueOf)
    .collect(Collectors.toCollection(ArrayList::new));
</code>```

Body:
I am trying to process a stream of Integers and collect the integers that match a predicate (via the compare() function) into a list. Here's a rough outline of the code I've written.

private List<Integer> process() {
    Z z = f(-1);
    return IntStream.range(0, 10)
        .filter(i -> compare(z, f(i)))
        .collect(Collectors.toCollection(ArrayList::new)); // Error on this line
}

private boolean compare(Z z1, Z z2) { ... }
private Z f(int i) { ... }

Unfortunately my solution does not compile and I cannot make sense of the compiler error for the line highlighted:

The method collect(Supplier<R>, ObjIntConsumer<R>, BiConsumer<R,R>) in the type IntStream is not applicable for the arguments (Collector<Object,capture#1-of ?,Collection<Object>>)

Any suggestions?

Body:
IntStream doesn't contain a collect method that accepts a single argument of type Collector. Stream does. Therefore you have to convert your IntStream to a Stream of objects.

You can either box the IntStream into a Stream<Integer> or use mapToObj to achieve the same.

For example:

return IntStream.range(0, 10)
    .filter(i -> compare(z, f(i)))
    .boxed()
    .collect(Collectors.toCollection(ArrayList::new));

boxed() will return a Stream consisting of the elements of this stream, each boxed to an Integer.

or

return IntStream.range(0, 10)
    .filter(i -> compare(z, f(i)))
    .mapToObj(Integer::valueOf)
    .collect(Collectors.toCollection(ArrayList::new));


问:
Convert OAuth token generated with Ruby to usable by Python
This question refers to youtube, specifically youtube analytics API and OAuth flow.

Using a ruby server-side webapp, I have created a token.

I now want to use that token in a client-side python app.

I have the client_secrets.json that generated the token.

I have generated tokens with python before and the format does not match that of the ruby generated token.

Is there existing code or a simple way to convert the ruby formatted token for use in python?  Yes, they are both json but the structures are different.

https://developers.google.com/youtube/reporting/guides/authorization/server-side-web-apps

答:
Assuming that you use the same client id and client secret for both ruby and python.   The refresh token that you got from one will work in the other.

Raw Oauth response:

```<code>{
"access_token" : "ya29.1.AADtN_VSBMC2Ga2lhxsTKjVQ_ROco8VbD6h01aj4PcKHLm6qvHbNtn-_BIzXMw",
"token_type" : "Bearer",
"expires_in" : 3600,
"refresh_token" : "1/J-3zPA8XR1o_cXebV9sDKn_f5MTqaFhKFxH-3PUPiJ4"
}
</code>```

Assuming you are using the client libraries then the issue you are going to have is how the different libraries store there credentials and read from them.   You are going to have to make your own parser for that.

Body:
Assuming that you use the same client id and client secret for both ruby and python.   The refresh token that you got from one will work in the other.

Raw Oauth response:

{
"access_token" : "ya29.1.AADtN_VSBMC2Ga2lhxsTKjVQ_ROco8VbD6h01aj4PcKHLm6qvHbNtn-_BIzXMw",
"token_type" : "Bearer",
"expires_in" : 3600,
"refresh_token" : "1/J-3zPA8XR1o_cXebV9sDKn_f5MTqaFhKFxH-3PUPiJ4"
}

Assuming you are using the client libraries then the issue you are going to have is how the different libraries store there credentials and read from them.   You are going to have to make your own parser for that.


问:
Save data to core data - Swift
When I save data to Core Data , duplicate data aren't save in one row.
When I save new data , one new row appears. That's happen error ( not exact data result ) display. How to handle Duplicate data in Core Data.</p>

答:
To save some data into CoreData you should use this example
Code swift 3.0</p>

```<code>let appDelegate = UIApplication.shared.delegate as! AppDelegate
    let context: NSManagedObjectContext = appDelegate.persistentContainer.viewContext

    let user = NSEntityDescription.insertNewObject(forEntityName: "Example", into: context)
    user.setValue("GOODDUDE", forKey: "name")

    appDelegate.saveContext()
</code>```

Title: Save data to core data - Swift

Body:
To save some data into CoreData you should use this example
Code swift 3.0

let appDelegate = UIApplication.shared.delegate as! AppDelegate
    let context: NSManagedObjectContext = appDelegate.persistentContainer.viewContext

    let user = NSEntityDescription.insertNewObject(forEntityName: "Example", into: context)
    user.setValue("GOODDUDE", forKey: "name")

    appDelegate.saveContext()


问:
why is maximum single-line length limited to 132 characters in Fortran Standard?
Is it about performance, clean source code, compilers, ...? I know that many compilers allow longer single-line codes. But, if this extension is possible without any compromise, then why does Fortran standard strictly adhere to this rule?

I know that this is very general question (stackoverflow warns me that this question might be downvoted given its title), but I cannot find any resources that explain the logic behind a max length of 132 characters in modern Fortran standard.

Update Oct 22, 2019: See https://j3-fortran.org/doc/year/19/19-138r1.txt for a proposal accepted as a work item for the next 202X revision of the Fortran standard, which eliminates the maximum line length and continuation limits.

答: In the olden days the punchcards also were of finite length. I forget what was being used for terminals in the 90s other than they were long CRTs, but do not recall the resolution... But it was NOT 2k pixels wide.</p>


问:
softlink to binary always use home folder path (instead of current folder)
kdevelop provides this AppImage binary:

```<code>wget -O KDevelop.AppImage https://download.kde.org/stable/kdevelop/5.1.1/bin/linux/KDevelop-5.1.1-x86_64.AppImage
chmod +x KDevelop.AppImage
./KDevelop.AppImage
</code>```

It works well. So I want to make a soft link called kd to that binary in /usr/bin, eg:

```<code>/usr/bin/sudo ln -s KDevelop-5.1.1-x86_64.AppImage kd
</code>```

Now if I run kd file1, I'd expect that it would open a file name file1 in the current folder, but it always tries to open a file name file1 in my home folder - which is not where it should be.

Is there some way to fix this issue?

答:
Some possible causes:

The application always assumes that you want to open files in your home directory, effectively or literally prepending $HOME to the path. This would be a bug in any *nix program, and should be reported.

The application behaves differently when $(basename "$0") is not KDevelop.AppImage (what @Scheff said).

You are actually running a different kd.

Possible workarounds/investigations:

Pass the full path to the file on the command line. If it tries to open /home/you//full/path/you/provided it is obviously buggy, and you have a test case. If it does not, then there might be some gotcha to what your $PWD actually is. Try checking its value before running.

Symlink with the same name, using sudo ln -s KDevelop-5.1.1-x86_64.AppImage /usr/bin, and try running that. If it behaves the same, you've at least proven that the symlink is not the problem.

Run type -a kd and verify that your /usr/bin/kd comes up first. If not there might be an alias or shell built-in which takes precedence.

That said, what is the actual error message?

Body:
kdevelop provides this AppImage binary:

wget -O KDevelop.AppImage https://download.kde.org/stable/kdevelop/5.1.1/bin/linux/KDevelop-5.1.1-x86_64.AppImage
chmod +x KDevelop.AppImage
./KDevelop.AppImage

It works well. So I want to make a soft link called kd to that binary in /usr/bin, eg:

/usr/bin/sudo ln -s KDevelop-5.1.1-x86_64.AppImage kd

Now if I run kd file1, I'd expect that it would open a file name file1 in the current folder, but it always tries to open a file name file1 in my home folder - which is not where it should be.

Is there some way to fix this issue?


问:
Appcelerator Studio: Cannot create a new project
I'm experiencing problems with Appcelerator Studio.
If I try to make a new project inside the workspace, I receive the following error:
"Error while reading/writing the .project file"

I tried to write outside the workspace. It doesn't create a Hello World project! Instead I have an empty project with only 3 files: CHANGELOG, LICENSE, manifest. Unable to add new models, views or controllers: the New command does nothing.

Any suggestions? Thanks in advance!

答:
I find entering:
appc logout
appc login

Fixes a lot of problems.  That being said I did have to uninstall/reinstall studio a couple of weeks ago as it wouldn't start up. I have also had goofy issues once when installed a node.js version above what was supported.


问:
I have a Objective C function that takes any type of object by reference. But when i pass a NSMutableArray My function does not recognise It
I have a function  That takes by reference any kind of object
-(BOOL)RemoteCall:(id**)DataClass;
in the implementation i use [*DataClass isMemberOfClass:[NSMutableArray class] to find out the type of the object. The problem is it does not work with NSMUtableArrays Does anybody have a solution to this problem ? Here is the relevant code:

Implementation:

```<code>-(BOOL)RemoteCall:(id**)DataClass
{
    if([*DataClass isMemberOfClass:[NSMutableArray class] ] == YES)
    {
            NSMutableArray * SW =(NSMutableArray *)*DataClass;
                       //do something with SW
            DataClass= (id**)SW;
                        return TRUE;
    }
}
</code>```

Any help and I mean anything at all will be appreciated, I'm stuck.

```<code>Method Call:
NSMutableArray * channelArray = [[NSMutableArray alloc]init]
Services * serv = [[Services alloc] init];
return [serv RemoteCall:&channelArray];
</code>```

答:
If you don't need to reassign your variable, then don't use this. id or NSObject * is just fine and works by reference anyway. id * or NSObject ** would be references. id ** doesn't make sense at all here.

Also, learn naming conventions (like upper/lowercase).

Body:
I have a function  That takes by reference any kind of object
-(BOOL)RemoteCall:(id**)DataClass;
in the implementation i use [*DataClass isMemberOfClass:[NSMutableArray class] to find out the type of the object. The problem is it does not work with NSMUtableArrays Does anybody have a solution to this problem ? Here is the relevant code:

Implementation:

-(BOOL)RemoteCall:(id**)DataClass
{
    if([*DataClass isMemberOfClass:[NSMutableArray class] ] == YES)
    {
            NSMutableArray * SW =(NSMutableArray *)*DataClass;
                       //do something with SW
            DataClass= (id**)SW;
                        return TRUE;
    }
}

Any help and I mean anything at all will be appreciated, I'm stuck.

Method Call:
NSMutableArray * channelArray = [[NSMutableArray alloc]init]
Services * serv = [[Services alloc] init];
return [serv RemoteCall:&channelArray];


问:
how to scrape through drop down menu using requests
I want to scrape through web pages that are selected from drop down as shown below

```<code> <h1>Scraping Test</h1>
    <form action="/tests/scraping" method='post'>
        <input type="hidden" name="csrf_token" value="1499585369##d2d1570f820aec0589b3bd5f4ab4e7df913e25ff"/>
        <table>
            <tr>
                <td>Select Ward: </td>
                <td>
                    <select name="ward">
                        <option value=''>-- select --</option>
                        <option value='DHANLAXMICOMPLEX'>DHANLAXMICOMPLEX</option>
                        <option value='POTALIYA'>POTALIYA</option>
                        <option value='ARJUNTOWER'>ARJUNTOWER</option>
                        <option value='NEWCLOTHMARKET'>NEWCLOTHMARKET</option>
                        <option value='CHANAKYAPURI'>CHANAKYAPURI</option>
                        <option value='BHAIKAKANAGAR'>BHAIKAKANAGAR</option>
                        <option value='RADHASWAMYROAD'>RADHASWAMYROAD</option>
                        <option value='SATADHAR'>SATADHAR</option>
                        <option value='AMRUTAVIDYALAYA'>AMRUTAVIDYALAYA</option>
                        <option value='AGARWALTOWERS'>AGARWALTOWERS</option>
                        <option value='RANNAPARK'>RANNAPARK</option>
                        <option value='IIM'>IIM</option>
                        <option value='VEJALPURWARD'>VEJALPURWARD</option>
                        <option value='GITAMANDIR'>GITAMANDIR</option>
                    </select>
                </td>
                <td><input type="submit" value="Search" class="search"/></td>
            </tr>
        </table>
</code>```

how to request a webpage from that drop down menu and there is also a search button
my code

```<code>import requests, csv
from lxml import html

def get_all_pages():

   payload = {'value':'DHANLAXMICOMPLEX'}
   url = requests.get('https://recruitment.advarisk.com/tests/scraping',data=payload)
   print(url.text)
</code>```

答:
There is a token value which you can get from this HTML element

```<code><input type="hidden" name="csrf_token" value="1499585369##d2d1570f820aec0589b3bd5f4ab4e7df913e25ff"/>
</code>```

and use in your request. Try to use below code and let me know in case of any issues

```<code>import lxml.html
import requests

url = "https://recruitment.advarisk.com/tests/scraping"
s = requests.session()
r = s.get(url)
source = lxml.html.document_fromstring(r.content)
token = source.xpath('//input[@name="csrf_token"]/@value')[0]

headers = {'Referer': 'https://recruitment.advarisk.com/tests/scraping'}
data = {'csrf_token': token, 'ward': 'DHANLAXMICOMPLEX'}

print(s.post(url, data=data, headers=headers).text)
</code>```

Body:
I want to scrape through web pages that are selected from drop down as shown below

Scraping Test

    <form action="/tests/scraping" method='post'>
        <input type="hidden" name="csrf_token" value="1499585369##d2d1570f820aec0589b3bd5f4ab4e7df913e25ff"/>

                Select Ward:

                    <select name="ward">
                        <option value=''>-- select --</option>
                        <option value='DHANLAXMICOMPLEX'>DHANLAXMICOMPLEX</option>
                        <option value='POTALIYA'>POTALIYA</option>
                        <option value='ARJUNTOWER'>ARJUNTOWER</option>
                        <option value='NEWCLOTHMARKET'>NEWCLOTHMARKET</option>
                        <option value='CHANAKYAPURI'>CHANAKYAPURI</option>
                        <option value='BHAIKAKANAGAR'>BHAIKAKANAGAR</option>
                        <option value='RADHASWAMYROAD'>RADHASWAMYROAD</option>
                        <option value='SATADHAR'>SATADHAR</option>
                        <option value='AMRUTAVIDYALAYA'>AMRUTAVIDYALAYA</option>
                        <option value='AGARWALTOWERS'>AGARWALTOWERS</option>
                        <option value='RANNAPARK'>RANNAPARK</option>
                        <option value='IIM'>IIM</option>
                        <option value='VEJALPURWARD'>VEJALPURWARD</option>
                        <option value='GITAMANDIR'>GITAMANDIR</option>
                    </select>

                <input type="submit" value="Search" class="search"/>

how to request a webpage from that drop down menu and there is also a search button
my code

import requests, csv
from lxml import html

def get_all_pages():

   payload = {'value':'DHANLAXMICOMPLEX'}
   url = requests.get('https://recruitment.advarisk.com/tests/scraping',data=payload)
   print(url.text)

Body:
There is a token value which you can get from this HTML element

<input type="hidden" name="csrf_token" value="1499585369##d2d1570f820aec0589b3bd5f4ab4e7df913e25ff"/>

and use in your request. Try to use below code and let me know in case of any issues

import lxml.html
import requests

url = "https://recruitment.advarisk.com/tests/scraping"
s = requests.session()
r = s.get(url)
source = lxml.html.document_fromstring(r.content)
token = source.xpath('//input[@name="csrf_token"]/@value')[0]

headers = {'Referer': 'https://recruitment.advarisk.com/tests/scraping'}
data = {'csrf_token': token, 'ward': 'DHANLAXMICOMPLEX'}

print(s.post(url, data=data, headers=headers).text)


问:
org.postgresql.util.PSQLException: ERROR: relation "bug_report_model" does not exist
I'm trying to run a simple java play 2.5 app on heroku. I am able to open the home page which generally means that connection to the database is successful. But when ever i try to call the route to add a dummy value to the table it shows

```<code>play.api.UnexpectedException: Unexpected exception[PersistenceException: ERROR executing DML bindLog[] error[ERROR: relation "bug_report_model" does not exist\n   Position: 13]]
</code>```

The code to add a value to the database is

```<code> public Result enterDummyTextInDb() {
    BugReportModel model = new BugReportModel();
    model.save();
    return ok(Json.toJson(model));
}
</code>```

The Model class is

```<code>@Entity
public class BugReportModel extends Model {
@Id
private Long id;
private Long timestamp = System.currentTimeMillis();

public BugReportModel() {
}

public static Finder<Long, BugReportModel> find = new Model.Finder<Long, BugReportModel>(BugReportModel.class);

public static List<BugReportModel> findAll() {
    return BugReportModel.find.orderBy("id").findList();
}

public Long getId() {
    return id;
}

public void setId(Long id) {
    this.id = id;
}

public Long getTimestamp() {
    return timestamp;
}

public void setTimestamp(Long timestamp) {
    this.timestamp = timestamp;
}
</code>```

}

found something for spring/hibernate that said that the table isnt created and so we have to add

```<code>spring.jpa.hibernate.ddl-auto=create
</code>```

but not sure about it for java play 2.5

答:
You need to configure your app to run evolutions by making your Procfile like this:

```<code>web: target/universal/stage/bin/myapp -Dhttp.port=${PORT} -Dplay.evolutions.db.default.autoApply=true -Ddb.default.driver=org.postgresql.Driver -Ddb.default.url=${DATABASE_URL}
</code>```

For more info see https://www.playframework.com/documentation/2.5.x/ProductionHeroku

Body:
I'm trying to run a simple java play 2.5 app on heroku. I am able to open the home page which generally means that connection to the database is successful. But when ever i try to call the route to add a dummy value to the table it shows

play.api.UnexpectedException: Unexpected exception[PersistenceException: ERROR executing DML bindLog[] error[ERROR: relation "bug_report_model" does not exist\n   Position: 13]]

The code to add a value to the database is

 public Result enterDummyTextInDb() {
    BugReportModel model = new BugReportModel();
    model.save();
    return ok(Json.toJson(model));
}

The Model class is

@Entity
public class BugReportModel extends Model {
@Id
private Long id;
private Long timestamp = System.currentTimeMillis();

public BugReportModel() {
}

public static Finder<Long, BugReportModel> find = new Model.Finder<Long, BugReportModel>(BugReportModel.class);

public static List<BugReportModel> findAll() {
    return BugReportModel.find.orderBy("id").findList();
}

public Long getId() {
    return id;
}

public void setId(Long id) {
    this.id = id;
}

public Long getTimestamp() {
    return timestamp;
}

public void setTimestamp(Long timestamp) {
    this.timestamp = timestamp;
}

}

found something for spring/hibernate that said that the table isnt created and so we have to add

spring.jpa.hibernate.ddl-auto=create

but not sure about it for java play 2.5

Body:
You need to configure your app to run evolutions by making your Procfile like this:

web: target/universal/stage/bin/myapp -Dhttp.port=${PORT} -Dplay.evolutions.db.default.autoApply=true -Ddb.default.driver=org.postgresql.Driver -Ddb.default.url=${DATABASE_URL}

For more info see https://www.playframework.com/documentation/2.5.x/ProductionHeroku


问:
Laravel Guzzle parsing
I am communicating with external API with Guzzle and fetching a response in this manner:

return $res->getBody()->getContents();

This part works fine, and I can see in Chrome inspector values returned. The issue I'm facing is fetching a specific value in the response. I have nested JSON with 3 values: aggregations (array of values), hits (array of values), total (number). I don't want to return everything, I just want to return values under hits.

$res->hits doesn't work

$res['hits'] doesn't work

json_encode($res) just returns an ugly long string

答:
```<code>$data = $res->getBody()->getContents();
return json_decode($data)->hits;
</code>```

Title: Laravel Guzzle parsing


问:
Add variable to nested list
Using R base, I would like to add a variable to a nested list where the variable changes for each nested list element. Below is an example. Thank you.</p>

```<code>#CREATE EXAMPLE DATAFRAME
DF <- expand.grid(NAME = c("FRANK", "TONY", "ED"), DATE = seq(as.Date("2014-01-01"), as.Date("2018-12-31"), by = "day"))

#CREATE NESTED LIST
DF <- lapply(1:3, function(i) lapply(2014:2015, function(t) DF[with(DF, as.Date(paste(t, "01", "01", sep = "-")) <= DATE & DATE <= as.Date(paste(t + i, "12", "31", sep = "-"))), ]))

#PRINT NESTED LIST
lapply(DF, lapply, function(x) rbind(head(x), tail(x)))

#I WOULD LIKE TO SIMPLIFY THIS PART
DF[[1]][[1]] <- within(DF[[1]][[1]], GROUP <- 2014)
DF[[1]][[2]] <- within(DF[[1]][[2]], GROUP <- 2015)

DF[[2]][[1]] <- within(DF[[2]][[1]], GROUP <- 2014)
DF[[2]][[2]] <- within(DF[[2]][[2]], GROUP <- 2015)

DF[[3]][[1]] <- within(DF[[3]][[1]], GROUP <- 2014)
DF[[3]][[2]] <- within(DF[[3]][[2]], GROUP <- 2015)

#PRINT MODIFIED NESTED LIST
lapply(DF, lapply, function(x) rbind(head(x), tail(x)))

#I AM SURPRISED THE FOLLOWING DOES NOT WORK
DF <- expand.grid(NAME = c("FRANK", "TONY", "ED"), DATE = seq(as.Date("2014-01-01"), as.Date("2018-12-31"), by = "day"))
DF <- lapply(1:3, function(i) lapply(2014:2015, function(t) DF[with(DF, as.Date(paste(t, "01", "01", sep = "-")) <= DATE & DATE <= as.Date(paste(t + i, "12", "31", sep = "-"))),]))
DF <- lapply(DF, function(x) lapply(2014:2015, function(t) within(x, GROUP <- t)))
lapply(DF, lapply, function(x) rbind(head(x), tail(x)))
</code>```

答:
This can be also achieved using Map in order to cbind the dates. i.e.,

```<code>lapply(DF, function(i) Map(cbind, i, c(2014, 2015)))

#or to set the name of that column to 'id',

lapply(DF, function(i) Map(function(x, y) cbind(x, id = y), i, c(2014, 2015)))
</code>```

However, If you don't mind flattening that list and keeping an extra id variable, you can use tidyverse to flatten and create the year id by graping the year from the first element of DATE from each data frame, i.e.

```<code>library(tidyverse)

new_df <- DF %>%
  flatten_df(.id = 'list_id') %>%
  group_by(list_id) %>%
  mutate(id = sub('-.*', '', DATE[1]))

#which will give,

# A tibble: 19,725 x 4
# Groups:   list_id [6]
#   list_id   NAME       DATE    id
#     <chr> <fctr>     <date> <chr>
# 1       1  FRANK 2014-01-01  2014
# 2       1   TONY 2014-01-01  2014
# 3       1     ED 2014-01-01  2014
# 4       1  FRANK 2014-01-02  2014
# 5       1   TONY 2014-01-02  2014
# 6       1     ED 2014-01-02  2014
# 7       1  FRANK 2014-01-03  2014
# 8       1   TONY 2014-01-03  2014
# 9       1     ED 2014-01-03  2014
#10       1  FRANK 2014-01-04  2014
# ... with 19,715 more rows
</code>```

Title: Add variable to nested list

Body:
Using R base, I would like to add a variable to a nested list where the variable changes for each nested list element. Below is an example. Thank you.

#CREATE EXAMPLE DATAFRAME
DF <- expand.grid(NAME = c("FRANK", "TONY", "ED"), DATE = seq(as.Date("2014-01-01"), as.Date("2018-12-31"), by = "day"))

#CREATE NESTED LIST
DF <- lapply(1:3, function(i) lapply(2014:2015, function(t) DF[with(DF, as.Date(paste(t, "01", "01", sep = "-")) <= DATE & DATE <= as.Date(paste(t + i, "12", "31", sep = "-"))), ]))

#PRINT NESTED LIST
lapply(DF, lapply, function(x) rbind(head(x), tail(x)))

#I WOULD LIKE TO SIMPLIFY THIS PART
DF[[1]][[1]] <- within(DF[[1]][[1]], GROUP <- 2014)
DF[[1]][[2]] <- within(DF[[1]][[2]], GROUP <- 2015)

DF[[2]][[1]] <- within(DF[[2]][[1]], GROUP <- 2014)
DF[[2]][[2]] <- within(DF[[2]][[2]], GROUP <- 2015)

DF[[3]][[1]] <- within(DF[[3]][[1]], GROUP <- 2014)
DF[[3]][[2]] <- within(DF[[3]][[2]], GROUP <- 2015)

#PRINT MODIFIED NESTED LIST
lapply(DF, lapply, function(x) rbind(head(x), tail(x)))

#I AM SURPRISED THE FOLLOWING DOES NOT WORK
DF <- expand.grid(NAME = c("FRANK", "TONY", "ED"), DATE = seq(as.Date("2014-01-01"), as.Date("2018-12-31"), by = "day"))
DF <- lapply(1:3, function(i) lapply(2014:2015, function(t) DF[with(DF, as.Date(paste(t, "01", "01", sep = "-")) <= DATE & DATE <= as.Date(paste(t + i, "12", "31", sep = "-"))),]))
DF <- lapply(DF, function(x) lapply(2014:2015, function(t) within(x, GROUP <- t)))
lapply(DF, lapply, function(x) rbind(head(x), tail(x)))

Body:
This can be also achieved using Map in order to cbind the dates. i.e.,

lapply(DF, function(i) Map(cbind, i, c(2014, 2015)))

#or to set the name of that column to 'id',

lapply(DF, function(i) Map(function(x, y) cbind(x, id = y), i, c(2014, 2015)))

However, If you don't mind flattening that list and keeping an extra id variable, you can use tidyverse to flatten and create the year id by graping the year from the first element of DATE from each data frame, i.e.

library(tidyverse)

new_df <- DF %>%
  flatten_df(.id = 'list_id') %>%
  group_by(list_id) %>%
  mutate(id = sub('-.*', '', DATE[1]))

#which will give,

# A tibble: 19,725 x 4
# Groups:   list_id [6]
#   list_id   NAME       DATE    id
#     <chr> <fctr>     <date> <chr>
# 1       1  FRANK 2014-01-01  2014
# 2       1   TONY 2014-01-01  2014
# 3       1     ED 2014-01-01  2014
# 4       1  FRANK 2014-01-02  2014
# 5       1   TONY 2014-01-02  2014
# 6       1     ED 2014-01-02  2014
# 7       1  FRANK 2014-01-03  2014
# 8       1   TONY 2014-01-03  2014
# 9       1     ED 2014-01-03  2014
#10       1  FRANK 2014-01-04  2014
# ... with 19,715 more rows


问:
What's the usage of a a def in a function?
Just reading the Python doc's in particular this page, in regards to attrgetter and itemgetter.

I'm struggling to understand the usage of the usage of def and return in the code examples. Not the usage of those functions.

It seems like a lamdba, but I don't know how to interpret it.

EDIT

I think it just clicked. The attrgetter is like a constructor for the resolve_attr function? This was a little confusing, as i'd normally expect the code for a def to be between the def and the return. The code on the return is like a lambda. To add to the confusion the arg ojb is the argument passed to the returned object. Arhhhhh.....

What do i need to know for this to make sense?

This is the code example I'm referring to:

```<code>   def attrgetter(*items):
    if any(not isinstance(item, str) for item in items):
        raise TypeError('attribute name must be a string')
    if len(items) == 1:
        attr = items[0]
        def g(obj): #What is this
            return resolve_attr(obj, attr)
    else:
        def g(obj):#And this?
            return tuple(resolve_attr(obj, attr) for attr in items)
    return g

def resolve_attr(obj, attr):
    for name in attr.split("."):
        obj = getattr(obj, name)
    return obj
</code>```

答:
In Python, functions are just like normal object. You can pass them around, and (as in the example above) return them from other functions.

The example function, returns another function as its return value. It does so by defining a function (that's the def g(ojb): line) and then simply returning it.

You can do this, because in Python - a function is just like any other object.

Body:
Just reading the Python doc's in particular this page, in regards to attrgetter and itemgetter.

I'm struggling to understand the usage of the usage of def and return in the code examples. Not the usage of those functions.

It seems like a lamdba, but I don't know how to interpret it.

EDIT

I think it just clicked. The attrgetter is like a constructor for the resolve_attr function? This was a little confusing, as i'd normally expect the code for a def to be between the def and the return. The code on the return is like a lambda. To add to the confusion the arg ojb is the argument passed to the returned object. Arhhhhh.....

What do i need to know for this to make sense?

This is the code example I'm referring to:

   def attrgetter(*items):
    if any(not isinstance(item, str) for item in items):
        raise TypeError('attribute name must be a string')
    if len(items) == 1:
        attr = items[0]
        def g(obj): #What is this
            return resolve_attr(obj, attr)
    else:
        def g(obj):#And this?
            return tuple(resolve_attr(obj, attr) for attr in items)
    return g

def resolve_attr(obj, attr):
    for name in attr.split("."):
        obj = getattr(obj, name)
    return obj


问:
Modal that opens an IFRAME (umbraco)
I'v got a problem when opening a Modal window that loads a JFrame.
The file loaded in the JFrame is a aspx file with a tag of /umbraco/controls/Tree/TreeControl.ascx. The modal loads and any text i put in is loaded but the TreeControl will not load. Iv used this ascx file in other places and it works greate if I load it via a normal page call. If you ar not familiar with umbraco Tree. It's basiclly a jquery tree that loads xml nodes.

Tried to copy code here. But couldnt get formatting to work.

Thx for any help
Best Regards
Marthin

答: This is no longer a problem for me with the new version of Umbraco 4.5</p>

Title: Modal that opens an IFRAME (umbraco)


问:
cmake ExternalProject_Add how to download URL non-tar file?
How does one set up an External project to download a link that isn't a .tgz file?

For example Catch provides a release that is a single header distribution.  I would like to just download this rather than the git repo or the .tgz release.  But I haven't figured out how to tell CMake to do this.

I would like to do something like:

```<code>Include(ExternalProject)
ExternalProject_Add(
    catch
    PREFIX ${CMAKE_BINARY_DIR}/catch
    URL https://github.com/philsquared/Catch/releases/download/v1.9.6/catch.hpp ${CMAKE_BINARY_DIR}/catch
    CONFIGURE_COMMAND ""
    BUILD_COMMAND ""
    INSTALL_COMMAND ""
    LOG_DOWNLOAD ON
)
</code>```

This fails as URL assumes that url is a compressed tar file.  I have tried various variants of the DOWNLOAD_COMMAND without any success either.

答:
Use DOWNLOAD_NO_EXTRACT option of ExternalProject_Add:

Just download the file and do not extract it; the full path to the downloaded file is available as <DOWNLOADED_FILE>.

Code example:

```<code>ExternalProject_Add(
    catch
    PREFIX ${CMAKE_BINARY_DIR}/catch
    URL https://github.com/philsquared/Catch/releases/download/v1.9.6/catch.hpp
    DOWNLOAD_NO_EXTRACT 1
    CONFIGURE_COMMAND ""
    BUILD_COMMAND ""
    INSTALL_COMMAND ""
    )
</code>```

File will be downloaded into <prefix>/src directory.

If use LOG_DOWNLOAD option alongside with DOWNLOAD_NO_EXTRACT, you need CMake 3.9 for work correctly. See these bugreports: https://gitlab.kitware.com/cmake/cmake/issues/16544, https://gitlab.kitware.com/cmake/cmake/issues/17046.

Body:
How does one set up an External project to download a link that isn't a .tgz file?

For example Catch provides a release that is a single header distribution.  I would like to just download this rather than the git repo or the .tgz release.  But I haven't figured out how to tell CMake to do this.

I would like to do something like:

Include(ExternalProject)
ExternalProject_Add(
    catch
    PREFIX ${CMAKE_BINARY_DIR}/catch
    URL https://github.com/philsquared/Catch/releases/download/v1.9.6/catch.hpp ${CMAKE_BINARY_DIR}/catch
    CONFIGURE_COMMAND ""
    BUILD_COMMAND ""
    INSTALL_COMMAND ""
    LOG_DOWNLOAD ON
)

This fails as URL assumes that url is a compressed tar file.  I have tried various variants of the DOWNLOAD_COMMAND without any success either.

Body:
Use DOWNLOAD_NO_EXTRACT option of ExternalProject_Add:

Just download the file and do not extract it; the full path to the downloaded file is available as <DOWNLOADED_FILE>.

Code example:

ExternalProject_Add(
    catch
    PREFIX ${CMAKE_BINARY_DIR}/catch
    URL https://github.com/philsquared/Catch/releases/download/v1.9.6/catch.hpp
    DOWNLOAD_NO_EXTRACT 1
    CONFIGURE_COMMAND ""
    BUILD_COMMAND ""
    INSTALL_COMMAND ""
    )

File will be downloaded into <prefix>/src directory.

If use LOG_DOWNLOAD option alongside with DOWNLOAD_NO_EXTRACT, you need CMake 3.9 for work correctly. See these bugreports: https://gitlab.kitware.com/cmake/cmake/issues/16544, https://gitlab.kitware.com/cmake/cmake/issues/17046.