问:
group_by operation in dplyr vs data.table for fast implementation
```<code>dat <- data.frame(yearID = rep(1:10000, each = 12),
                  monthID = rep(1:12, times = 10000),
                  x1 = rnorm(120000),
                  x2 = rnorm(120000),
                  x3 = rnorm(120000),
                  x4 = rnorm(120000),
                  x5 = rnorm(120000),
                  x6 = rnorm(120000),
                  p.start = 6,
                  p.end = 7,
                  m.start = 8,
                  m.end = 9,
                  h.start = 10,
                  h.end = 11)
</code>```

I need to do some operations on the above data which is described below after my current solution

```<code>library(dplyr)

start_time <- Sys.time()

df1 <- dat %>%
       tidyr::gather(., index_name, value, x1:x6) %>%
       dplyr::filter(!index_name %in% c('x5','x6')) %>%
       dplyr::group_by(yearID, index_name) %>%
       dplyr::summarise(p.start.val = sum(value[monthID == p.start]),
                        p.val = sum(value[monthID >= p.start & monthID <= p.end]),
                        m.val = sum(value[monthID >= m.start & monthID <= m.end]),
                        h.val = sum(value[monthID >= h.start & monthID <= h.end]),
                        h.end.val = sum(value[monthID == h.end])) %>%
       tidyr::gather(., variable, value, p.start.val:h.end.val) %>%
       dplyr::mutate(new.col.name = paste0(index_name,'_',variable)) %>%
       dplyr::select(-index_name, -variable) %>%
       tidyr::spread(., new.col.name, value) %>%
       dplyr::mutate(yearRef = 2018)

colnames(df1) <-  sub(".val", "", colnames(df1))

df2 <- dat %>%
       tidyr::gather(., index_name, value, x1:x6) %>%
       dplyr::filter(index_name %in% c('x4','x6')) %>%
       dplyr::group_by(yearID, index_name) %>%
       dplyr::summarise(p.end.val = value[monthID == p.end],
                        m.end.val = value[monthID == m.end],
                        h.end.val = value[monthID == h.end]) %>%
       tidyr::gather(., variable, value, p.end.val:h.end.val) %>%
       dplyr::mutate(new.col.name = paste0(index_name,'_',variable)) %>%
       dplyr::select(-index_name, -variable) %>%
       tidyr::spread(., new.col.name, value) %>%
       dplyr::mutate(yearRef = 2018)

colnames(df2) <-  sub(".val", "", colnames(df2))

final.dat <- Reduce(function(...) merge(..., by = c( "yearID", "yearRef"), all.x=TRUE), list(df1,df2))

 end_time <- Sys.time()

 end_time - start_time

 # Time difference of 2.054761 secs
</code>```

What I want to do is:

for variable x1 to x4, I want to sum them for different months as shown in df1

for variable x5 to x6, I want to select the values for selected month for each year as shown in df2

My code above works fine but takes quite a time if the size of dat increases i.e. if number of years become 20000 instead of 10000. I am wondering if someone could help me with a data.table to implement the above solution which I hope would make this faster. Thank you.

答:
I'll run this on df1 only, since from there the pattern is easily repeatable.

Notes:

I'm using magrittr solely to help break out each step in the chain, as each of dplyr's *verbsare directly translatable. It is not difficult to convert this into a non-magrittr` pipeline. The benefit of using it (as is also the case with tidyverse pipes) is, in my opinion, readability and therefore maintainability.

The Answer

I'll walk through the steps below.

library(data.table)
library(magrittr)

as.data.table(dat) %>%
  melt(., measure.vars = grep("^x[0-9]+", colnames(.)),
       variable.name = "index_name", variable.factor = FALSE) %>%
  .[ !index_name %in% c("x5", "x6"), ] %>%
  .[, .(
    p.start.val = sum(value[monthID == p.start]),
    p.val = sum(value[monthID >= p.start & monthID <= p.end]),
    m.val = sum(value[monthID >= m.start & monthID <= m.end]),
    h.val = sum(value[monthID >= h.start & monthID <= h.end]),
    h.end.val = sum(value[monthID == h.end])
  ), by = .(yearID, index_name) ] %>%
  melt(., id.vars = 1:2, variable.factor = FALSE) %>%
  .[, new.col.name := paste0(index_name, "_", variable) ] %>%
  .[, c("index_name", "variable") := NULL ] %>%
  dcast(., yearID ~ new.col.name) %>%
  .[, yearRef := 2018 ]
```

<hr>

<h2>Steps:</h2>

<p>Notes for the steps:</p>

<ul>
<li><p>In the walk-through, I add <code>dplyr::arrange_all()</code> and <code>.[order(.),]</code> to the end of each intermediate pipe just so that we have apples-to-apples comparisons.</p></li>
<li><p>You did not include a random seed for your sample. I used <code>set.seed(42)</code>, so to compare your console with what I'm showing, you'll need to set this seed and regenerate <code>dat</code>.</p></li>
<li><p>Each code block continues from the previous step's code, I shorten all repeated code to <code>... %>%</code> for brevity to make this answer much less voluminous.</p></li>
</ul>

<p>The steps:</p>

<ol>
<li><p><code>tidyr::gather</code> to <code>data.table::melt</code>. There's likely a better way than <code>grep</code> to select ranges of columns in <code>data.table::melt</code>, but while <code>as.data.table(dat)[, -(x1:x6)]</code> works as one might infer, the same column-ranging does not work within <code>melt</code>.</p>

<pre class="lang-r prettyprint-override"><code>dat %>%
  tidyr::gather(., index_name, value, x1:x6) %>%
  arrange_all() %>% head() # just for comparison
# # A tibble: 6 x 10
#   yearID monthID p.start p.end m.start m.end h.start h.end index_name  value
#    <int>   <int>   <dbl> <dbl>   <dbl> <dbl>   <dbl> <dbl> <chr>       <dbl>
# 1      1       1       6     7       8     9      10    11 x1          1.37
# 2      1       1       6     7       8     9      10    11 x2         -0.483
# 3      1       1       6     7       8     9      10    11 x3         -0.314
# 4      1       1       6     7       8     9      10    11 x4         -2.23
# 5      1       1       6     7       8     9      10    11 x5         -0.717
# 6      1       1       6     7       8     9      10    11 x6         -1.04
as.data.table(dat) %>%
  melt(., measure.vars = grep("^x[0-9]+", colnames(.)),
       variable.name = "index_name", variable.factor = FALSE) %>%
  .[order(.),] %>% head() # just for comparison
#    yearID monthID p.start p.end m.start m.end h.start h.end index_name      value
# 1:      1       1       6     7       8     9      10    11         x1  1.3709584
# 2:      1       1       6     7       8     9      10    11         x2 -0.4831687
# 3:      1       1       6     7       8     9      10    11         x3 -0.3139498
# 4:      1       1       6     7       8     9      10    11         x4 -2.2323282
# 5:      1       1       6     7       8     9      10    11         x5 -0.7167575
# 6:      1       1       6     7       8     9      10    11         x6 -1.0357630

</code>```

add in dplyr::filter and dplyr::summarise (grouped); I literally just copied the new variables' assignments from summarise(...) into .( ... ) block, no change was necessary.

... %>%
  dplyr::filter(!index_name %in% c('x5','x6')) %>%
  dplyr::group_by(yearID, index_name) %>%
  dplyr::summarise(p.start.val = sum(value[monthID == p.start]),
                   p.val = sum(value[monthID >= p.start & monthID <= p.end]),
                   m.val = sum(value[monthID >= m.start & monthID <= m.end]),
                   h.val = sum(value[monthID >= h.start & monthID <= h.end]),
                   h.end.val = sum(value[monthID == h.end])) %>%
  arrange_all() %>% head() # just for comparison
# # A tibble: 6 x 7
# # Groups:   yearID [2]
#   yearID index_name p.start.val  p.val   m.val  h.val h.end.val
#    <int> <chr>            <dbl>  <dbl>   <dbl>  <dbl>     <dbl>
# 1      1 x1             -0.106   1.41   1.92    1.24      1.30
# 2      1 x2              0.573  -0.516 -2.29   -3.54     -0.990
# 3      1 x3              0.767   0.455  0.461   2.28      2.08
# 4      1 x4             -0.0559 -1.11  -0.0975 -0.326    -0.483
# 5      2 x1             -2.66   -5.10   1.01   -1.95     -0.172
# 6      2 x2              0.342  -0.546  0.605   1.51      1.25
... %>%
  .[ !index_name %in% c("x5", "x6"), ] %>%
  .[, .(
    p.start.val = sum(value[monthID == p.start]),
    p.val = sum(value[monthID >= p.start & monthID <= p.end]),
    m.val = sum(value[monthID >= m.start & monthID <= m.end]),
    h.val = sum(value[monthID >= h.start & monthID <= h.end]),
    h.end.val = sum(value[monthID == h.end])
  ), by = .(yearID, index_name) ] %>%
  .[order(.),] %>% head(.) # just for comparison
#    yearID index_name p.start.val      p.val       m.val      h.val  h.end.val
# 1:      1         x1 -0.10612452  1.4053975  1.92376468  1.2421556  1.3048697
# 2:      1         x2  0.57306337 -0.5164756 -2.28861552 -3.5367198 -0.9901743
# 3:      1         x3  0.76706512  0.4546020  0.46096277  2.2819246  2.0842981
# 4:      1         x4 -0.05589648 -1.1093361 -0.09748514 -0.3260778 -0.4825699
# 5:      2         x1 -2.65645542 -5.0969223  1.01347475 -1.9532258 -0.1719174
# 6:      2         x2  0.34227065 -0.5457969  0.60537738  1.5136450  1.2498633
```</li>
<li><p><code>tidyr::gather</code> again</p>

<pre class="lang-r prettyprint-override"><code>... %>%
  tidyr::gather(., variable, value, p.start.val:h.end.val) %>%
  arrange_all() %>% head() # just for comparison
# # A tibble: 6 x 4
# # Groups:   yearID [1]
#   yearID index_name variable     value
#    <int> <chr>      <chr>        <dbl>
# 1      1 x1         h.end.val    1.30
# 2      1 x1         h.val        1.24
# 3      1 x1         m.val        1.92
# 4      1 x1         p.start.val -0.106
# 5      1 x1         p.val        1.41
# 6      1 x2         h.end.val   -0.990
... %>%
  melt(., id.vars = 1:2, variable.factor = FALSE) %>%
  .[order(.),] %>% head(.) # just for comparison
#    yearID index_name    variable      value
# 1:      1         x1   h.end.val  1.3048697
# 2:      1         x1       h.val  1.2421556
# 3:      1         x1       m.val  1.9237647
# 4:      1         x1 p.start.val -0.1061245
# 5:      1         x1       p.val  1.4053975
# 6:      1         x2   h.end.val -0.9901743
</code>```

tidyr::spread to data.table::dcast

... %>%
  dplyr::mutate(new.col.name = paste0(index_name,'_',variable)) %>%
  dplyr::select(-index_name, -variable) %>%
  tidyr::spread(., new.col.name, value) %>%
  arrange_all() %>% head() # just for comparison
# # A tibble: 6 x 21
# # Groups:   yearID [6]
#   yearID x1_h.end.val x1_h.val x1_m.val x1_p.start.val x1_p.val x2_h.end.val x2_h.val x2_m.val x2_p.start.val x2_p.val x3_h.end.val x3_h.val x3_m.val x3_p.start.val x3_p.val x4_h.end.val x4_h.val x4_m.val x4_p.start.val x4_p.val
#    <int>        <dbl>    <dbl>    <dbl>          <dbl>    <dbl>        <dbl>    <dbl>    <dbl>          <dbl>    <dbl>        <dbl>    <dbl>    <dbl>          <dbl>    <dbl>        <dbl>    <dbl>    <dbl>          <dbl>    <dbl>
# 1      1        1.30     1.24     1.92          -0.106    1.41        -0.990   -3.54   -2.29            0.573   -0.516        2.08     2.28     0.461          0.767    0.455      -0.483   -0.326   -0.0975        -0.0559  -1.11
# 2      2       -0.172   -1.95     1.01          -2.66    -5.10         1.25     1.51    0.605           0.342   -0.546       -1.38    -0.731    0.443         -0.725   -1.17       -0.623   -1.91     1.49          -0.806   -0.717
# 3      3        0.505   -0.104    1.74          -0.640   -0.185        0.570    1.68   -2.24           -0.103   -1.02        -1.36    -2.50    -0.918          1.36     1.26        0.0847  -0.280    0.699          0.114   -0.582
# 4      4       -0.811   -0.379   -2.09          -0.361    0.397       -0.782    0.110  -0.0187         -0.641   -0.149       -1.47    -2.45    -1.27           0.418    0.131       0.0582   0.885    0.784          0.998   -0.0115
# 5      5       -2.99    -2.90     0.956          0.643    0.733        0.165    0.382   1.46            1.48     2.16        -0.451   -0.213   -0.357          0.222    0.686      -0.949   -0.156    1.23           1.35     0.908
# 6      6       -1.04    -0.322    1.96           1.30     1.64         0.838   -0.406   1.86            0.863    2.11         0.479    2.37    -1.13          -1.22    -1.63       -0.970    0.0391  -1.08           0.683   -1.24
... %>%
  .[, new.col.name := paste0(index_name, "_", variable) ] %>%
  .[, c("index_name", "variable") := NULL ] %>%
  dcast(., yearID ~ new.col.name) %>%
  .[order(.),] %>% head(.) # just for comparison
#    yearID x1_h.end.val   x1_h.val   x1_m.val x1_p.start.val   x1_p.val x2_h.end.val   x2_h.val    x2_m.val x2_p.start.val   x2_p.val x3_h.end.val   x3_h.val   x3_m.val x3_p.start.val   x3_p.val x4_h.end.val    x4_h.val    x4_m.val x4_p.start.val    x4_p.val
# 1:      1    1.3048697  1.2421556  1.9237647     -0.1061245  1.4053975   -0.9901743 -3.5367198 -2.28861552      0.5730634 -0.5164756    2.0842981  2.2819246  0.4609628      0.7670651  0.4546020  -0.48256993 -0.32607779 -0.09748514    -0.05589648 -1.10933614
# 2:      2   -0.1719174 -1.9532258  1.0134748     -2.6564554 -5.0969223    1.2498633  1.5136450  0.60537738      0.3422707 -0.5457969   -1.3790815 -0.7305400  0.4429124     -0.7249950 -1.1681343  -0.62293711 -1.90725766  1.48980773    -0.80634526 -0.71692479
# 3:      3    0.5049551 -0.1039713  1.7399409     -0.6399949 -0.1845448    0.5697303  1.6768675 -2.24285021     -0.1029872 -1.0245616   -1.3608773 -2.5029906 -0.9178704      1.3641160  1.2619892   0.08468983 -0.27967757  0.69899862     0.11429665 -0.58216791
# 4:      4   -0.8113932 -0.3785752 -2.0949859     -0.3610573  0.3971059   -0.7823128  0.1098614 -0.01867344     -0.6414615 -0.1488759   -1.4653210 -2.4476336 -1.2718183      0.4179297  0.1311655   0.05823201  0.88484095  0.78382293     0.99795594 -0.01147192
# 5:      5   -2.9930901 -2.9032572  0.9558396      0.6428993  0.7326600    0.1645109  0.3819658  1.45532687      1.4820236  2.1608213   -0.4513016 -0.2129462 -0.3572757      0.2221201  0.6855960  -0.94859958 -0.15646638  1.23051588     1.34645936  0.90755241
# 6:      6   -1.0431189 -0.3222408  1.9592347      1.3025426  1.6383908    0.8379162 -0.4059827  1.86142674      0.8626753  2.1076609    0.4792767  2.3683451 -1.1252801     -1.2213407 -1.6339743  -0.96979464  0.03912882 -1.08199221     0.68254513 -1.23950872
```</li>
<li><p>finish it up</p>

<pre class="lang-r prettyprint-override"><code>df1a <- df1 %>% arrange_all()
head(df1a)
# # A tibble: 6 x 22
# # Groups:   yearID [6]
#   yearID x1_h.end   x1_h   x1_m x1_p.start   x1_p x2_h.end   x2_h    x2_m x2_p.start   x2_p x3_h.end   x3_h   x3_m x3_p.start   x3_p x4_h.end    x4_h    x4_m x4_p.start    x4_p yearRef
#    <int>    <dbl>  <dbl>  <dbl>      <dbl>  <dbl>    <dbl>  <dbl>   <dbl>      <dbl>  <dbl>    <dbl>  <dbl>  <dbl>      <dbl>  <dbl>    <dbl>   <dbl>   <dbl>      <dbl>   <dbl>   <dbl>
# 1      1    1.30   1.24   1.92      -0.106  1.41    -0.990 -3.54  -2.29        0.573 -0.516    2.08   2.28   0.461      0.767  0.455  -0.483  -0.326  -0.0975    -0.0559 -1.11      2018
# 2      2   -0.172 -1.95   1.01      -2.66  -5.10     1.25   1.51   0.605       0.342 -0.546   -1.38  -0.731  0.443     -0.725 -1.17   -0.623  -1.91    1.49      -0.806  -0.717     2018
# 3      3    0.505 -0.104  1.74      -0.640 -0.185    0.570  1.68  -2.24       -0.103 -1.02    -1.36  -2.50  -0.918      1.36   1.26    0.0847 -0.280   0.699      0.114  -0.582     2018
# 4      4   -0.811 -0.379 -2.09      -0.361  0.397   -0.782  0.110 -0.0187     -0.641 -0.149   -1.47  -2.45  -1.27       0.418  0.131   0.0582  0.885   0.784      0.998  -0.0115    2018
# 5      5   -2.99  -2.90   0.956      0.643  0.733    0.165  0.382  1.46        1.48   2.16    -0.451 -0.213 -0.357      0.222  0.686  -0.949  -0.156   1.23       1.35    0.908     2018
# 6      6   -1.04  -0.322  1.96       1.30   1.64     0.838 -0.406  1.86        0.863  2.11     0.479  2.37  -1.13      -1.22  -1.63   -0.970   0.0391 -1.08       0.683  -1.24      2018
df1b <- ... %>%
  .[, yearRef := 2018 ] %>%
  .[order(.),]
head(df1b)
#    yearID x1_h.end.val   x1_h.val   x1_m.val x1_p.start.val   x1_p.val x2_h.end.val   x2_h.val    x2_m.val x2_p.start.val   x2_p.val x3_h.end.val   x3_h.val   x3_m.val x3_p.start.val   x3_p.val x4_h.end.val    x4_h.val    x4_m.val x4_p.start.val    x4_p.val yearRef
# 1:      1    1.3048697  1.2421556  1.9237647     -0.1061245  1.4053975   -0.9901743 -3.5367198 -2.28861552      0.5730634 -0.5164756    2.0842981  2.2819246  0.4609628      0.7670651  0.4546020  -0.48256993 -0.32607779 -0.09748514    -0.05589648 -1.10933614    2018
# 2:      2   -0.1719174 -1.9532258  1.0134748     -2.6564554 -5.0969223    1.2498633  1.5136450  0.60537738      0.3422707 -0.5457969   -1.3790815 -0.7305400  0.4429124     -0.7249950 -1.1681343  -0.62293711 -1.90725766  1.48980773    -0.80634526 -0.71692479    2018
# 3:      3    0.5049551 -0.1039713  1.7399409     -0.6399949 -0.1845448    0.5697303  1.6768675 -2.24285021     -0.1029872 -1.0245616   -1.3608773 -2.5029906 -0.9178704      1.3641160  1.2619892   0.08468983 -0.27967757  0.69899862     0.11429665 -0.58216791    2018
# 4:      4   -0.8113932 -0.3785752 -2.0949859     -0.3610573  0.3971059   -0.7823128  0.1098614 -0.01867344     -0.6414615 -0.1488759   -1.4653210 -2.4476336 -1.2718183      0.4179297  0.1311655   0.05823201  0.88484095  0.78382293     0.99795594 -0.01147192    2018
# 5:      5   -2.9930901 -2.9032572  0.9558396      0.6428993  0.7326600    0.1645109  0.3819658  1.45532687      1.4820236  2.1608213   -0.4513016 -0.2129462 -0.3572757      0.2221201  0.6855960  -0.94859958 -0.15646638  1.23051588     1.34645936  0.90755241    2018
# 6:      6   -1.0431189 -0.3222408  1.9592347      1.3025426  1.6383908    0.8379162 -0.4059827  1.86142674      0.8626753  2.1076609    0.4792767  2.3683451 -1.1252801     -1.2213407 -1.6339743  -0.96979464  0.03912882 -1.08199221     0.68254513 -1.23950872    2018
</code>```

They do match:

identical(as.data.frame(df1a), as.data.frame(df1b))
# [1] TRUE
```

<p>The speed-ups are not gargantuan, but they do appear to be significant. One way you can speed up your own code (still <code>dplyr</code>) is to remove the <code>group</code>ing as soon as you don't need it. If I add <code>ungroup()</code> immediately after <code>summarise(...)</code>, I see a small improvement.</p>

<pre class="lang-r prettyprint-override"><code>microbenchmark::microbenchmark(
  dplyr = { ... },
  dplyr_ungrp = { ... },
  data.table = { ... },
  times = 10
)
# Unit: milliseconds
#         expr      min        lq      mean    median        uq       max neval
#        dplyr 988.8311 1021.4725 1048.5462 1045.6885 1066.2733 1135.6032    10
#  dplyr_ungrp 909.3643  913.9301  952.6282  937.6540  998.2802 1041.2144    10
#   data.table 457.4500  465.1788  478.1471  474.2388  478.9840  531.1449    10
</code>```

Body:
dat <- data.frame(yearID = rep(1:10000, each = 12),
                  monthID = rep(1:12, times = 10000),
                  x1 = rnorm(120000),
                  x2 = rnorm(120000),
                  x3 = rnorm(120000),
                  x4 = rnorm(120000),
                  x5 = rnorm(120000),
                  x6 = rnorm(120000),
                  p.start = 6,
                  p.end = 7,
                  m.start = 8,
                  m.end = 9,
                  h.start = 10,
                  h.end = 11)

I need to do some operations on the above data which is described below after my current solution

library(dplyr)

start_time <- Sys.time()

df1 <- dat %>%
       tidyr::gather(., index_name, value, x1:x6) %>%
       dplyr::filter(!index_name %in% c('x5','x6')) %>%
       dplyr::group_by(yearID, index_name) %>%
       dplyr::summarise(p.start.val = sum(value[monthID == p.start]),
                        p.val = sum(value[monthID >= p.start & monthID <= p.end]),
                        m.val = sum(value[monthID >= m.start & monthID <= m.end]),
                        h.val = sum(value[monthID >= h.start & monthID <= h.end]),
                        h.end.val = sum(value[monthID == h.end])) %>%
       tidyr::gather(., variable, value, p.start.val:h.end.val) %>%
       dplyr::mutate(new.col.name = paste0(index_name,'_',variable)) %>%
       dplyr::select(-index_name, -variable) %>%
       tidyr::spread(., new.col.name, value) %>%
       dplyr::mutate(yearRef = 2018)

colnames(df1) <-  sub(".val", "", colnames(df1))

df2 <- dat %>%
       tidyr::gather(., index_name, value, x1:x6) %>%
       dplyr::filter(index_name %in% c('x4','x6')) %>%
       dplyr::group_by(yearID, index_name) %>%
       dplyr::summarise(p.end.val = value[monthID == p.end],
                        m.end.val = value[monthID == m.end],
                        h.end.val = value[monthID == h.end]) %>%
       tidyr::gather(., variable, value, p.end.val:h.end.val) %>%
       dplyr::mutate(new.col.name = paste0(index_name,'_',variable)) %>%
       dplyr::select(-index_name, -variable) %>%
       tidyr::spread(., new.col.name, value) %>%
       dplyr::mutate(yearRef = 2018)

colnames(df2) <-  sub(".val", "", colnames(df2))

final.dat <- Reduce(function(...) merge(..., by = c( "yearID", "yearRef"), all.x=TRUE), list(df1,df2))

 end_time <- Sys.time()

 end_time - start_time

 # Time difference of 2.054761 secs

What I want to do is:

for variable x1 to x4, I want to sum them for different months as shown in df1

for variable x5 to x6, I want to select the values for selected month for each year as shown in df2

My code above works fine but takes quite a time if the size of dat increases i.e. if number of years become 20000 instead of 10000. I am wondering if someone could help me with a data.table to implement the above solution which I hope would make this faster. Thank you.

Body:
I'll run this on df1 only, since from there the pattern is easily repeatable.

Notes:

I'm using magrittr solely to help break out each step in the chain, as each of dplyr's *verbsare directly translatable. It is not difficult to convert this into a non-magrittr` pipeline. The benefit of using it (as is also the case with tidyverse pipes) is, in my opinion, readability and therefore maintainability.

The Answer

I'll walk through the steps below.

library(data.table)
library(magrittr)

as.data.table(dat) %>%
  melt(., measure.vars = grep("^x[0-9]+", colnames(.)),
       variable.name = "index_name", variable.factor = FALSE) %>%
  .[ !index_name %in% c("x5", "x6"), ] %>%
  .[, .(
    p.start.val = sum(value[monthID == p.start]),
    p.val = sum(value[monthID >= p.start & monthID <= p.end]),
    m.val = sum(value[monthID >= m.start & monthID <= m.end]),
    h.val = sum(value[monthID >= h.start & monthID <= h.end]),
    h.end.val = sum(value[monthID == h.end])
  ), by = .(yearID, index_name) ] %>%
  melt(., id.vars = 1:2, variable.factor = FALSE) %>%
  .[, new.col.name := paste0(index_name, "_", variable) ] %>%
  .[, c("index_name", "variable") := NULL ] %>%
  dcast(., yearID ~ new.col.name) %>%
  .[, yearRef := 2018 ]

Steps:

Notes for the steps:

In the walk-through, I add dplyr::arrange_all() and .[order(.),] to the end of each intermediate pipe just so that we have apples-to-apples comparisons.

You did not include a random seed for your sample. I used set.seed(42), so to compare your console with what I'm showing, you'll need to set this seed and regenerate dat.

Each code block continues from the previous step's code, I shorten all repeated code to ... %>% for brevity to make this answer much less voluminous.

The steps:

tidyr::gather to data.table::melt. There's likely a better way than grep to select ranges of columns in data.table::melt, but while as.data.table(dat)[, -(x1:x6)] works as one might infer, the same column-ranging does not work within melt.

dat %>%
  tidyr::gather(., index_name, value, x1:x6) %>%
  arrange_all() %>% head() # just for comparison
# # A tibble: 6 x 10
#   yearID monthID p.start p.end m.start m.end h.start h.end index_name  value
#    <int>   <int>   <dbl> <dbl>   <dbl> <dbl>   <dbl> <dbl> <chr>       <dbl>
# 1      1       1       6     7       8     9      10    11 x1          1.37
# 2      1       1       6     7       8     9      10    11 x2         -0.483
# 3      1       1       6     7       8     9      10    11 x3         -0.314
# 4      1       1       6     7       8     9      10    11 x4         -2.23
# 5      1       1       6     7       8     9      10    11 x5         -0.717
# 6      1       1       6     7       8     9      10    11 x6         -1.04
as.data.table(dat) %>%
  melt(., measure.vars = grep("^x[0-9]+", colnames(.)),
       variable.name = "index_name", variable.factor = FALSE) %>%
  .[order(.),] %>% head() # just for comparison
#    yearID monthID p.start p.end m.start m.end h.start h.end index_name      value
# 1:      1       1       6     7       8     9      10    11         x1  1.3709584
# 2:      1       1       6     7       8     9      10    11         x2 -0.4831687
# 3:      1       1       6     7       8     9      10    11         x3 -0.3139498
# 4:      1       1       6     7       8     9      10    11         x4 -2.2323282
# 5:      1       1       6     7       8     9      10    11         x5 -0.7167575
# 6:      1       1       6     7       8     9      10    11         x6 -1.0357630

add in dplyr::filter and dplyr::summarise (grouped); I literally just copied the new variables' assignments from summarise(...) into .( ... ) block, no change was necessary.

... %>%
  dplyr::filter(!index_name %in% c('x5','x6')) %>%
  dplyr::group_by(yearID, index_name) %>%
  dplyr::summarise(p.start.val = sum(value[monthID == p.start]),
                   p.val = sum(value[monthID >= p.start & monthID <= p.end]),
                   m.val = sum(value[monthID >= m.start & monthID <= m.end]),
                   h.val = sum(value[monthID >= h.start & monthID <= h.end]),
                   h.end.val = sum(value[monthID == h.end])) %>%
  arrange_all() %>% head() # just for comparison
# # A tibble: 6 x 7
# # Groups:   yearID [2]
#   yearID index_name p.start.val  p.val   m.val  h.val h.end.val
#    <int> <chr>            <dbl>  <dbl>   <dbl>  <dbl>     <dbl>
# 1      1 x1             -0.106   1.41   1.92    1.24      1.30
# 2      1 x2              0.573  -0.516 -2.29   -3.54     -0.990
# 3      1 x3              0.767   0.455  0.461   2.28      2.08
# 4      1 x4             -0.0559 -1.11  -0.0975 -0.326    -0.483
# 5      2 x1             -2.66   -5.10   1.01   -1.95     -0.172
# 6      2 x2              0.342  -0.546  0.605   1.51      1.25
... %>%
  .[ !index_name %in% c("x5", "x6"), ] %>%
  .[, .(
    p.start.val = sum(value[monthID == p.start]),
    p.val = sum(value[monthID >= p.start & monthID <= p.end]),
    m.val = sum(value[monthID >= m.start & monthID <= m.end]),
    h.val = sum(value[monthID >= h.start & monthID <= h.end]),
    h.end.val = sum(value[monthID == h.end])
  ), by = .(yearID, index_name) ] %>%
  .[order(.),] %>% head(.) # just for comparison
#    yearID index_name p.start.val      p.val       m.val      h.val  h.end.val
# 1:      1         x1 -0.10612452  1.4053975  1.92376468  1.2421556  1.3048697
# 2:      1         x2  0.57306337 -0.5164756 -2.28861552 -3.5367198 -0.9901743
# 3:      1         x3  0.76706512  0.4546020  0.46096277  2.2819246  2.0842981
# 4:      1         x4 -0.05589648 -1.1093361 -0.09748514 -0.3260778 -0.4825699
# 5:      2         x1 -2.65645542 -5.0969223  1.01347475 -1.9532258 -0.1719174
# 6:      2         x2  0.34227065 -0.5457969  0.60537738  1.5136450  1.2498633

tidyr::gather again

... %>%
  tidyr::gather(., variable, value, p.start.val:h.end.val) %>%
  arrange_all() %>% head() # just for comparison
# # A tibble: 6 x 4
# # Groups:   yearID [1]
#   yearID index_name variable     value
#    <int> <chr>      <chr>        <dbl>
# 1      1 x1         h.end.val    1.30
# 2      1 x1         h.val        1.24
# 3      1 x1         m.val        1.92
# 4      1 x1         p.start.val -0.106
# 5      1 x1         p.val        1.41
# 6      1 x2         h.end.val   -0.990
... %>%
  melt(., id.vars = 1:2, variable.factor = FALSE) %>%
  .[order(.),] %>% head(.) # just for comparison
#    yearID index_name    variable      value
# 1:      1         x1   h.end.val  1.3048697
# 2:      1         x1       h.val  1.2421556
# 3:      1         x1       m.val  1.9237647
# 4:      1         x1 p.start.val -0.1061245
# 5:      1         x1       p.val  1.4053975
# 6:      1         x2   h.end.val -0.9901743

tidyr::spread to data.table::dcast

... %>%
  dplyr::mutate(new.col.name = paste0(index_name,'_',variable)) %>%
  dplyr::select(-index_name, -variable) %>%
  tidyr::spread(., new.col.name, value) %>%
  arrange_all() %>% head() # just for comparison
# # A tibble: 6 x 21
# # Groups:   yearID [6]
#   yearID x1_h.end.val x1_h.val x1_m.val x1_p.start.val x1_p.val x2_h.end.val x2_h.val x2_m.val x2_p.start.val x2_p.val x3_h.end.val x3_h.val x3_m.val x3_p.start.val x3_p.val x4_h.end.val x4_h.val x4_m.val x4_p.start.val x4_p.val
#    <int>        <dbl>    <dbl>    <dbl>          <dbl>    <dbl>        <dbl>    <dbl>    <dbl>          <dbl>    <dbl>        <dbl>    <dbl>    <dbl>          <dbl>    <dbl>        <dbl>    <dbl>    <dbl>          <dbl>    <dbl>
# 1      1        1.30     1.24     1.92          -0.106    1.41        -0.990   -3.54   -2.29            0.573   -0.516        2.08     2.28     0.461          0.767    0.455      -0.483   -0.326   -0.0975        -0.0559  -1.11
# 2      2       -0.172   -1.95     1.01          -2.66    -5.10         1.25     1.51    0.605           0.342   -0.546       -1.38    -0.731    0.443         -0.725   -1.17       -0.623   -1.91     1.49          -0.806   -0.717
# 3      3        0.505   -0.104    1.74          -0.640   -0.185        0.570    1.68   -2.24           -0.103   -1.02        -1.36    -2.50    -0.918          1.36     1.26        0.0847  -0.280    0.699          0.114   -0.582
# 4      4       -0.811   -0.379   -2.09          -0.361    0.397       -0.782    0.110  -0.0187         -0.641   -0.149       -1.47    -2.45    -1.27           0.418    0.131       0.0582   0.885    0.784          0.998   -0.0115
# 5      5       -2.99    -2.90     0.956          0.643    0.733        0.165    0.382   1.46            1.48     2.16        -0.451   -0.213   -0.357          0.222    0.686      -0.949   -0.156    1.23           1.35     0.908
# 6      6       -1.04    -0.322    1.96           1.30     1.64         0.838   -0.406   1.86            0.863    2.11         0.479    2.37    -1.13          -1.22    -1.63       -0.970    0.0391  -1.08           0.683   -1.24
... %>%
  .[, new.col.name := paste0(index_name, "_", variable) ] %>%
  .[, c("index_name", "variable") := NULL ] %>%
  dcast(., yearID ~ new.col.name) %>%
  .[order(.),] %>% head(.) # just for comparison
#    yearID x1_h.end.val   x1_h.val   x1_m.val x1_p.start.val   x1_p.val x2_h.end.val   x2_h.val    x2_m.val x2_p.start.val   x2_p.val x3_h.end.val   x3_h.val   x3_m.val x3_p.start.val   x3_p.val x4_h.end.val    x4_h.val    x4_m.val x4_p.start.val    x4_p.val
# 1:      1    1.3048697  1.2421556  1.9237647     -0.1061245  1.4053975   -0.9901743 -3.5367198 -2.28861552      0.5730634 -0.5164756    2.0842981  2.2819246  0.4609628      0.7670651  0.4546020  -0.48256993 -0.32607779 -0.09748514    -0.05589648 -1.10933614
# 2:      2   -0.1719174 -1.9532258  1.0134748     -2.6564554 -5.0969223    1.2498633  1.5136450  0.60537738      0.3422707 -0.5457969   -1.3790815 -0.7305400  0.4429124     -0.7249950 -1.1681343  -0.62293711 -1.90725766  1.48980773    -0.80634526 -0.71692479
# 3:      3    0.5049551 -0.1039713  1.7399409     -0.6399949 -0.1845448    0.5697303  1.6768675 -2.24285021     -0.1029872 -1.0245616   -1.3608773 -2.5029906 -0.9178704      1.3641160  1.2619892   0.08468983 -0.27967757  0.69899862     0.11429665 -0.58216791
# 4:      4   -0.8113932 -0.3785752 -2.0949859     -0.3610573  0.3971059   -0.7823128  0.1098614 -0.01867344     -0.6414615 -0.1488759   -1.4653210 -2.4476336 -1.2718183      0.4179297  0.1311655   0.05823201  0.88484095  0.78382293     0.99795594 -0.01147192
# 5:      5   -2.9930901 -2.9032572  0.9558396      0.6428993  0.7326600    0.1645109  0.3819658  1.45532687      1.4820236  2.1608213   -0.4513016 -0.2129462 -0.3572757      0.2221201  0.6855960  -0.94859958 -0.15646638  1.23051588     1.34645936  0.90755241
# 6:      6   -1.0431189 -0.3222408  1.9592347      1.3025426  1.6383908    0.8379162 -0.4059827  1.86142674      0.8626753  2.1076609    0.4792767  2.3683451 -1.1252801     -1.2213407 -1.6339743  -0.96979464  0.03912882 -1.08199221     0.68254513 -1.23950872

finish it up

df1a <- df1 %>% arrange_all()
head(df1a)
# # A tibble: 6 x 22
# # Groups:   yearID [6]
#   yearID x1_h.end   x1_h   x1_m x1_p.start   x1_p x2_h.end   x2_h    x2_m x2_p.start   x2_p x3_h.end   x3_h   x3_m x3_p.start   x3_p x4_h.end    x4_h    x4_m x4_p.start    x4_p yearRef
#    <int>    <dbl>  <dbl>  <dbl>      <dbl>  <dbl>    <dbl>  <dbl>   <dbl>      <dbl>  <dbl>    <dbl>  <dbl>  <dbl>      <dbl>  <dbl>    <dbl>   <dbl>   <dbl>      <dbl>   <dbl>   <dbl>
# 1      1    1.30   1.24   1.92      -0.106  1.41    -0.990 -3.54  -2.29        0.573 -0.516    2.08   2.28   0.461      0.767  0.455  -0.483  -0.326  -0.0975    -0.0559 -1.11      2018
# 2      2   -0.172 -1.95   1.01      -2.66  -5.10     1.25   1.51   0.605       0.342 -0.546   -1.38  -0.731  0.443     -0.725 -1.17   -0.623  -1.91    1.49      -0.806  -0.717     2018
# 3      3    0.505 -0.104  1.74      -0.640 -0.185    0.570  1.68  -2.24       -0.103 -1.02    -1.36  -2.50  -0.918      1.36   1.26    0.0847 -0.280   0.699      0.114  -0.582     2018
# 4      4   -0.811 -0.379 -2.09      -0.361  0.397   -0.782  0.110 -0.0187     -0.641 -0.149   -1.47  -2.45  -1.27       0.418  0.131   0.0582  0.885   0.784      0.998  -0.0115    2018
# 5      5   -2.99  -2.90   0.956      0.643  0.733    0.165  0.382  1.46        1.48   2.16    -0.451 -0.213 -0.357      0.222  0.686  -0.949  -0.156   1.23       1.35    0.908     2018
# 6      6   -1.04  -0.322  1.96       1.30   1.64     0.838 -0.406  1.86        0.863  2.11     0.479  2.37  -1.13      -1.22  -1.63   -0.970   0.0391 -1.08       0.683  -1.24      2018
df1b <- ... %>%
  .[, yearRef := 2018 ] %>%
  .[order(.),]
head(df1b)
#    yearID x1_h.end.val   x1_h.val   x1_m.val x1_p.start.val   x1_p.val x2_h.end.val   x2_h.val    x2_m.val x2_p.start.val   x2_p.val x3_h.end.val   x3_h.val   x3_m.val x3_p.start.val   x3_p.val x4_h.end.val    x4_h.val    x4_m.val x4_p.start.val    x4_p.val yearRef
# 1:      1    1.3048697  1.2421556  1.9237647     -0.1061245  1.4053975   -0.9901743 -3.5367198 -2.28861552      0.5730634 -0.5164756    2.0842981  2.2819246  0.4609628      0.7670651  0.4546020  -0.48256993 -0.32607779 -0.09748514    -0.05589648 -1.10933614    2018
# 2:      2   -0.1719174 -1.9532258  1.0134748     -2.6564554 -5.0969223    1.2498633  1.5136450  0.60537738      0.3422707 -0.5457969   -1.3790815 -0.7305400  0.4429124     -0.7249950 -1.1681343  -0.62293711 -1.90725766  1.48980773    -0.80634526 -0.71692479    2018
# 3:      3    0.5049551 -0.1039713  1.7399409     -0.6399949 -0.1845448    0.5697303  1.6768675 -2.24285021     -0.1029872 -1.0245616   -1.3608773 -2.5029906 -0.9178704      1.3641160  1.2619892   0.08468983 -0.27967757  0.69899862     0.11429665 -0.58216791    2018
# 4:      4   -0.8113932 -0.3785752 -2.0949859     -0.3610573  0.3971059   -0.7823128  0.1098614 -0.01867344     -0.6414615 -0.1488759   -1.4653210 -2.4476336 -1.2718183      0.4179297  0.1311655   0.05823201  0.88484095  0.78382293     0.99795594 -0.01147192    2018
# 5:      5   -2.9930901 -2.9032572  0.9558396      0.6428993  0.7326600    0.1645109  0.3819658  1.45532687      1.4820236  2.1608213   -0.4513016 -0.2129462 -0.3572757      0.2221201  0.6855960  -0.94859958 -0.15646638  1.23051588     1.34645936  0.90755241    2018
# 6:      6   -1.0431189 -0.3222408  1.9592347      1.3025426  1.6383908    0.8379162 -0.4059827  1.86142674      0.8626753  2.1076609    0.4792767  2.3683451 -1.1252801     -1.2213407 -1.6339743  -0.96979464  0.03912882 -1.08199221     0.68254513 -1.23950872    2018

They do match:

identical(as.data.frame(df1a), as.data.frame(df1b))
# [1] TRUE

The speed-ups are not gargantuan, but they do appear to be significant. One way you can speed up your own code (still dplyr) is to remove the grouping as soon as you don't need it. If I add ungroup() immediately after summarise(...), I see a small improvement.

microbenchmark::microbenchmark(
  dplyr = { ... },
  dplyr_ungrp = { ... },
  data.table = { ... },
  times = 10
)
# Unit: milliseconds
#         expr      min        lq      mean    median        uq       max neval
#        dplyr 988.8311 1021.4725 1048.5462 1045.6885 1066.2733 1135.6032    10
#  dplyr_ungrp 909.3643  913.9301  952.6282  937.6540  998.2802 1041.2144    10
#   data.table 457.4500  465.1788  478.1471  474.2388  478.9840  531.1449    10


来源: stackexchange

问:
SPA won't loads in 4G but loads in Wifi
I have an application (writen in React.js) that works perfectly in Chrome Browser on my computer.

When I try to load the app on my phone with Chrome, the app loads well when I am in Wifi but not when I am in my mobile network (4G).

I have a perfectly valid SSL certificate created with Let's Encrypt.

Do you have an idea why ?

答:
Did you try to use google chrome dev tool with 3G etc...

Please see the article : https://developers.google.com/web/tools/chrome-devtools/network

ContentLicense: CC BY-SA 4.0

Title: SPA won't loads in 4G but loads in Wifi


问:
Drake: A plant include Muti-body plant and LeafSystem plant
A part of a plant is easily modeled by MutibodyPlant Class, unfortunately, other parts of a plant look like to be modeled by leaf System. Can Drake do such things?
Plant A: muti-links arm --> Using SDF and MutibodyPlant class for modeling
Plant B: an inertial wheel is attached to the end of the arm --> Using Leaf system to define dynamic(time derivate) of an inertial wheel.   </p>

<p>An illustration for this question     </p>

答: I agree with Sherm.  But if you do have some non-rigid-body forces to apply, then you can certainly do what you’ve asked.  You just need to apply the “interaction forces” either through the actuation inputs or add a new spatial force input.</p>

Body:
A part of a plant is easily modeled by MutibodyPlant Class, unfortunately, other parts of a plant look like to be modeled by leaf System. Can Drake do such things?
Plant A: muti-links arm --> Using SDF and MutibodyPlant class for modeling
Plant B: an inertial wheel is attached to the end of the arm --> Using Leaf system to define dynamic(time derivate) of an inertial wheel.

An illustration for this question


问:
Image rotation in react-konva
```<code>const bush1Image = new Image();
bush1Image.src = 'bush1.png';
bush1Image.rotate(90);
</code>```

I need to rotate the image before using this.setState, so why the above code doesn't work?

答:
 element created with new Image() doesn't have rotate method.

If you want to rotate it you have two ways:

1 Just rotate created Konva image with

<Image image={this.state.image} rotation={90} />

2 Or draw your image into an external canvas with required rotation, and then use that canvas as image property for Konva nodes

const bush1Image = new Image();
bush1Image.onload = () => {
   const canvas = document.createElement('canvas');
   // reversed size, because image will be rotated
   canvas.width = bush1Image.height;
   canvas.height = bush1Image.width;
   const ctx = canvas.getContext('2d');
   ctx.moveTo(0, canvas.widht);
   ctx.rotate(90 * Math.PI / 180);
   ctx.drawImage(bush1Image, 0, 0);
   this.setState({ image: canvas })
}
bush1Image.src = 'bush1.png';

```

Title: Image rotation in react-konva

Body:
const bush1Image = new Image();
bush1Image.src = 'bush1.png';
bush1Image.rotate(90);

I need to rotate the image before using this.setState, so why the above code doesn't work?

Body:
<img> element created with new Image() doesn't have rotate method.

If you want to rotate it you have two ways:

1 Just rotate created Konva image with

<Image image={this.state.image} rotation={90} />

2 Or draw your image into an external canvas with required rotation, and then use that canvas as image property for Konva nodes

const bush1Image = new Image();
bush1Image.onload = () => {
   const canvas = document.createElement('canvas');
   // reversed size, because image will be rotated
   canvas.width = bush1Image.height;
   canvas.height = bush1Image.width;
   const ctx = canvas.getContext('2d');
   ctx.moveTo(0, canvas.widht);
   ctx.rotate(90 * Math.PI / 180);
   ctx.drawImage(bush1Image, 0, 0);
   this.setState({ image: canvas })
}
bush1Image.src = 'bush1.png';


问:
Flutter URL launcher Google Maps crashes App
A couple of days ago launching both Google Maps and Apple Maps perfectly fine. I updated Flutter to its newest Version while on master channel to 1.13.1. I assume after updating it stopped working. So I changed channels to beta and tried, but still not working. When trying to launch Google Maps or Apple Maps the IOS app crashes and I  neither understand the error nor did I find something on the internet.

Any help is much appreciated.

My UI code:

ListTile(
                 title: Text("Launch Google Maps", textAlign: TextAlign.center),
                  onTap: () async {
                    final String clubName =
                        clubs[index].name.toString().replaceAll(" ", "+");
                    print("Name: $clubName"); // 'Flawless+Club'
                    final String clubCity = clubs[index].stadt;
                    print("Name: $clubCity"); // 'London'
                    final String newGoogleUrl =
                        "https://www.google.com/maps/dir/?api=1&destination=$clubName+$clubCity";

                    if (await canLaunch(newGoogleUrl)) {
                      await launch(newGoogleUrl);
                    } else {
                      throw 'Could not launch $newGoogleUrl';
                    }
                  },
                ),
```

<p>Flutter doctor -v Ouput:</p>

```[✓] Flutter (Channel beta, v1.12.13+hotfix.3, on Mac OS X 10.15.1 19B88, locale en-DE)
    • Flutter version 1.12.13+hotfix.3 at /Library/Flutter
    • Framework revision 57f2df76d7 (2 days ago), 2019-12-05 21:23:21 -0800
    • Engine revision ac9391978e
    • Dart version 2.7.0

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at /Users/murat/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b01)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.2.1, Build version 11B500
    • CocoaPods version 1.8.4

[✓] Android Studio (version 3.4)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 35.3.1
    • Dart plugin version 183.6270
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b01)

[✓] VS Code (version 1.40.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.7.0

[✓] Connected device (1 available)

• No issues found!
```

<p>Error when tapping the ListTile:</p>

```Lost connection to device.
*** First throw call stack:
(
    0   CoreFoundation                      0x00007fff23c4f02e __exceptionPreprocess + 350
    1   libobjc.A.dylib                     0x00007fff50b97b20 objc_exception_throw + 48
    2   Foundation                          0x00007fff2578c55b -[__NSConcreteURLComponents initWithString:] + 0
    3   CoreServices                        0x00007fff24e970bd -[_LSURLOverride initWithOriginalURL:checkingForAvailableApplications:newsOnly:] + 151
    4   CoreServices                        0x00007fff24e97992 -[_LSURLOverride initWithOriginalURL:newsOnly:] + 25
    5   CoreServices                        0x00007fff24e982ae _ZN14LaunchServices12URLOverridesL20getURLOverrideCommonEP5NSURLb + 399
    6   CoreServices                        0x00007fff24e9810e -[LSApplicationWorkspace(LSURLOverride) URLOverrideForURL:] + 14
    7   UIKitCore<…>
Exited (sigterm)
```

答:
I had the same error.
My problem was that I was using this url which contain a name for the waypoint.

'https://maps.apple.com/?q=' + '$name' + '&ll=$lat,$lon';

If the waypoint name ($name) contains spaces like 'New York' launch_url is not converting the space character to %20.

I fixed it by using name.toString().replaceAll(' ', "%20").
Full url example:
https://maps.apple.com/?q=' + name.toString().replaceAll(' ', "%20") + '&ll=$lat,$lon'

Body:
A couple of days ago launching both Google Maps and Apple Maps perfectly fine. I updated Flutter to its newest Version while on master channel to 1.13.1. I assume after updating it stopped working. So I changed channels to beta and tried, but still not working. When trying to launch Google Maps or Apple Maps the IOS app crashes and I  neither understand the error nor did I find something on the internet.

Any help is much appreciated.

My UI code:

ListTile(
                 title: Text("Launch Google Maps", textAlign: TextAlign.center),
                  onTap: () async {
                    final String clubName =
                        clubs[index].name.toString().replaceAll(" ", "+");
                    print("Name: $clubName"); // 'Flawless+Club'
                    final String clubCity = clubs[index].stadt;
                    print("Name: $clubCity"); // 'London'
                    final String newGoogleUrl =
                        "https://www.google.com/maps/dir/?api=1&destination=$clubName+$clubCity";

                    if (await canLaunch(newGoogleUrl)) {
                      await launch(newGoogleUrl);
                    } else {
                      throw 'Could not launch $newGoogleUrl';
                    }
                  },
                ),

Flutter doctor -v Ouput:

[✓] Flutter (Channel beta, v1.12.13+hotfix.3, on Mac OS X 10.15.1 19B88, locale en-DE)
    • Flutter version 1.12.13+hotfix.3 at /Library/Flutter
    • Framework revision 57f2df76d7 (2 days ago), 2019-12-05 21:23:21 -0800
    • Engine revision ac9391978e
    • Dart version 2.7.0

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at /Users/murat/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b01)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.2.1, Build version 11B500
    • CocoaPods version 1.8.4

[✓] Android Studio (version 3.4)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 35.3.1
    • Dart plugin version 183.6270
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b01)

[✓] VS Code (version 1.40.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.7.0

[✓] Connected device (1 available)

• No issues found!

Error when tapping the ListTile:

Lost connection to device.
*** First throw call stack:
(
    0   CoreFoundation                      0x00007fff23c4f02e __exceptionPreprocess + 350
    1   libobjc.A.dylib                     0x00007fff50b97b20 objc_exception_throw + 48
    2   Foundation                          0x00007fff2578c55b -[__NSConcreteURLComponents initWithString:] + 0
    3   CoreServices                        0x00007fff24e970bd -[_LSURLOverride initWithOriginalURL:checkingForAvailableApplications:newsOnly:] + 151
    4   CoreServices                        0x00007fff24e97992 -[_LSURLOverride initWithOriginalURL:newsOnly:] + 25
    5   CoreServices                        0x00007fff24e982ae _ZN14LaunchServices12URLOverridesL20getURLOverrideCommonEP5NSURLb + 399
    6   CoreServices                        0x00007fff24e9810e -[LSApplicationWorkspace(LSURLOverride) URLOverrideForURL:] + 14
    7   UIKitCore<…>
Exited (sigterm)


问:
What's the preferred way to create a jQuery object with attributes?
When creating jQuery objects I've used the following syntax lately, as described here:

```<code>var $el = $('<div/>', {
   class: 'class-1 class-2'
});
</code>```

Safari 5.0.5 gives a syntax error at the point where I use the above construction.

Removing the second argument and adding classes with addClass removes the error, but seems rather unelegant.

How do you create your objects? I tried with attr({class: 'class-1'}), as well, but received the same syntax error.

答:
Since class is a reserved word, you need to put it in quotes.

```<code>var $el = $('<div/>', {
   "class": 'class-1 class-2'
});
</code>```

ContentLicense: CC BY-SA 3.0

Body:
When creating jQuery objects I've used the following syntax lately, as described here:

var $el = $('
', {
   class: 'class-1 class-2'
});

Safari 5.0.5 gives a syntax error at the point where I use the above construction.

Removing the second argument and adding classes with addClass removes the error, but seems rather unelegant.

How do you create your objects? I tried with attr({class: 'class-1'}), as well, but received the same syntax error.

Body:
Since class is a reserved word, you need to put it in quotes.

var $el = $('<div/>', {
   "class": 'class-1 class-2'
});


问:
Media queries running weird because of non-integer width
Can't say is it a real problem or I'm just being paranoid but this behavior of media queries is really driving me crazy for last couple of hours.

Let's consider this really simple CSS.

```<code> body {background:yellow}

 @media (max-width:399px) {
   body {background:red}
 }

 @media (min-width:400px) {
   body {background:blue}
 }
</code>```

Problem happens when width is 399.333px! (or any float-value between 399 and 400 integers)

My logic says that by using this CSS style page will never turn yellow, right? It should be red when viewport size is shorter than 400px in width and blue when it's 400px and over.

Weird behavior happens with Opera browser (I'm using 36.0 at the moment) on Windows when page is zoomed-in. I understand that viewport width is calculated using real viewport width and current zoom-level and this value should always be integer. But...

Looks like Opera doesn't round/floor/ceil that value which affects on entire page. I'm getting yellow background when Opera finds out that viewport-width is not 399px or 400px but it's 399.333px!? So none of media queries fulfills condition.

I've already tried to find an answer here and web-wide but nothing is close enough to this problem. This problem already happened to me when I was using em units so I could work around and turn them to pixels, but I can't affect user's decision about using browser's zoom feature.

Is there something I can do to prevent this or that's just the way it is?

The easiest way to simulate this behavior is hitting CTRL,+ three times and than easily move vertical slider in Object Inspector.

update:

Yes, I can fix it with "mobile/desktop first" approach by linking each media break-point to previous one but that's not part of my question. Also, default body style is here as visual aid only and changing that really doesn't solve problem.

答:
A simple solution could be the following:

```<code> body {background:yellow}

 @media (max-width:400px) {
   body {background:red}
 }

 @media (min-width:400px) {
   body {background:blue}
 }
</code>```

The rules in the last media query will simply overwrite any parameters that exist previously, just because of the order.

That way there won't be a situation/width which isn't covered by these two media queries:  Everything up to 399.9999... (whatever) fulfills the  first condition, everything above 400 will meet the second condition, and if the width is exactly 400, the rules in the second media query will overwrite the previous ones due to their order.

Body:
Can't say is it a real problem or I'm just being paranoid but this behavior of media queries is really driving me crazy for last couple of hours.

Let's consider this really simple CSS.

 body {background:yellow}

 @media (max-width:399px) {
   body {background:red}
 }

 @media (min-width:400px) {
   body {background:blue}
 }

Problem happens when width is 399.333px! (or any float-value between 399 and 400 integers)

My logic says that by using this CSS style page will never turn yellow, right? It should be red when viewport size is shorter than 400px in width and blue when it's 400px and over.

Weird behavior happens with Opera browser (I'm using 36.0 at the moment) on Windows when page is zoomed-in. I understand that viewport width is calculated using real viewport width and current zoom-level and this value should always be integer. But...

Looks like Opera doesn't round/floor/ceil that value which affects on entire page. I'm getting yellow background when Opera finds out that viewport-width is not 399px or 400px but it's 399.333px!? So none of media queries fulfills condition.

I've already tried to find an answer here and web-wide but nothing is close enough to this problem. This problem already happened to me when I was using em units so I could work around and turn them to pixels, but I can't affect user's decision about using browser's zoom feature.

Is there something I can do to prevent this or that's just the way it is?

The easiest way to simulate this behavior is hitting CTRL,+ three times and than easily move vertical slider in Object Inspector.

update:

Yes, I can fix it with "mobile/desktop first" approach by linking each media break-point to previous one but that's not part of my question. Also, default body style is here as visual aid only and changing that really doesn't solve problem.

Body:
A simple solution could be the following:

 body {background:yellow}

 @media (max-width:400px) {
   body {background:red}
 }

 @media (min-width:400px) {
   body {background:blue}
 }

The rules in the last media query will simply overwrite any parameters that exist previously, just because of the order.

That way there won't be a situation/width which isn't covered by these two media queries:  Everything up to 399.9999... (whatever) fulfills the  first condition, everything above 400 will meet the second condition, and if the width is exactly 400, the rules in the second media query will overwrite the previous ones due to their order.


问:
React JS setState for an array
I have a data and each data has a state, I put it on React Hooks array state. This is my current state that I had after requesting data from server

console.log("panduan : ", panduan)

// Output
panduan :
[6] -> [false, false, false, false, false, false]
```

<p>So each of these boolean are supposed to be a trigger button for this component below.</p>

<p>I have a component and each data should set their own state, this is my component.</p>

<pre class="lang-js prettyprint-override"><code>data.map((post, index) => {
  <TriggerButton onClick={() => setPanduan([!panduan[index]])}>
    {post.year}
  </TriggerButton>
})

// this Child component should be shown when state is true
const content = (
  <Triggered show={panduan[index]}>
    {post.someData}
  </Triggered>
)
</code>```

this code above is still not working, maybe because I didn't use the setPanduan on the right way. So how to setPanduan for each data?

答:
I think you may be looking for something more like this in your Parent onClick method

```<code>onClick={() => {
  const tempArray = [...panduan];
  tempArray[index] = !tempArray[index];
  setPanduan(tempArray);
}
</code>```

as what you currently have would end up changing the entire state to a single item array and remove the data that was initialised there from the server :)

Title: React JS setState for an array

Body:
I have a data and each data has a state, I put it on React Hooks array state. This is my current state that I had after requesting data from server

console.log("panduan : ", panduan)

// Output
panduan :
[6] -> [false, false, false, false, false, false]

So each of these boolean are supposed to be a trigger button for this component below.

I have a component and each data should set their own state, this is my component.

data.map((post, index) => {
  <TriggerButton onClick={() => setPanduan([!panduan[index]])}>
    {post.year}
  </TriggerButton>
})

// this Child component should be shown when state is true
const content = (
  <Triggered show={panduan[index]}>
    {post.someData}
  </Triggered>
)

this code above is still not working, maybe because I didn't use the setPanduan on the right way. So how to setPanduan for each data?

Body:
I think you may be looking for something more like this in your Parent onClick method

onClick={() => {
  const tempArray = [...panduan];
  tempArray[index] = !tempArray[index];
  setPanduan(tempArray);
}

as what you currently have would end up changing the entire state to a single item array and remove the data that was initialised there from the server :)


答:
You are not setting the updated array right. Make sure to destructure/clone the old array and toggle only the intended index like so:     </p>

```<code>data.map((post, index) => {
   <TriggerButton onClick={() => {
      // clone old state
      const newArray = [...panduan];
      // update toggled index
      newArray[index] = !newArray[index]
      setPanduan(newArray)
    }>
      {post.year}
   </TriggerButton>
})
</code>```

Title: React JS setState for an array

Body:
You are not setting the updated array right. Make sure to destructure/clone the old array and toggle only the intended index like so:

data.map((post, index) => {
   <TriggerButton onClick={() => {
      // clone old state
      const newArray = [...panduan];
      // update toggled index
      newArray[index] = !newArray[index]
      setPanduan(newArray)
    }>
      {post.year}
   </TriggerButton>
})


问:
want to download images from server
I want to transfer site from shared hosting server to dedicated server.current server has folder,which includes approx. 16000 images...we can not use FTP to download these much images.and i do not have SSH rights? how can i download images from this shared hosting server.</p>

答:
we can not use FTP to download these much images

Nonsense. FTP (the protocol) is perfectly capable of downloading 16000 files. If your FTP program is causing you trouble, simply pick a better FTP program. If you can handle commandline applications, wget is nice, since it supports recursion and continuation.

Title: want to download images from server


问:
Appium, WebDriver
I am new to Appium. I try to execute my selenium script in mobile using Appium. But I got an error like below

org.openqa.selenium.WebDriverException: Returned value cannot be
  converted to WebElement: {ELEMENT=0.5768692365381867-1}

Code I am using is as below :-

```<code>AppiumDriver d;
    @BeforeClass
    public void setUp() throws Exception
    {
        //Set Capabilities
        DesiredCapabilities caps=new DesiredCapabilities();
        caps.setCapability("deviceName", "Celkon A35K Remote");
        caps.setCapability("platformName", "Android");
        caps.setCapability("platformVersion", "4.4.2");
        caps.setCapability("browserName", "Browser");

        d = new AndroidDriver(new URL("http://0.0.0.0:4723/wd/hub"), caps);
        d.manage().timeouts().implicitlyWait(60,TimeUnit.SECONDS);
        d.manage().timeouts().pageLoadTimeout(120,TimeUnit.SECONDS);
    }
@Test
public void testFBLogin() throws Exception
{
    //Load web page
    d.get("https://www.facebook.com/");
    //Enter user name
    d.findElementByName("email").sendKeys("Appium");
    //Enter password
    d.findElementByName("pass").sendKeys("Appium");
    //Click on Login
    d.findElementByName("login").click();
    Thread.sleep(10000);
}
@AfterClass
public void tearDown()
{
    //Close browser
    d.quit();
}`script:
</code>```

答:
You need to modify your AppiumDriver variable to:

AppiumDriver d;

This way you are telling Appium you are searching for a WebElement.

Title: Appium, WebDriver

Body:
I am new to Appium. I try to execute my selenium script in mobile using Appium. But I got an error like below

org.openqa.selenium.WebDriverException: Returned value cannot be
  converted to WebElement: {ELEMENT=0.5768692365381867-1}

Code I am using is as below :-

AppiumDriver d;
    @BeforeClass
    public void setUp() throws Exception
    {
        //Set Capabilities
        DesiredCapabilities caps=new DesiredCapabilities();
        caps.setCapability("deviceName", "Celkon A35K Remote");
        caps.setCapability("platformName", "Android");
        caps.setCapability("platformVersion", "4.4.2");
        caps.setCapability("browserName", "Browser");

        d = new AndroidDriver(new URL("http://0.0.0.0:4723/wd/hub"), caps);
        d.manage().timeouts().implicitlyWait(60,TimeUnit.SECONDS);
        d.manage().timeouts().pageLoadTimeout(120,TimeUnit.SECONDS);
    }
@Test
public void testFBLogin() throws Exception
{
    //Load web page
    d.get("https://www.facebook.com/");
    //Enter user name
    d.findElementByName("email").sendKeys("Appium");
    //Enter password
    d.findElementByName("pass").sendKeys("Appium");
    //Click on Login
    d.findElementByName("login").click();
    Thread.sleep(10000);
}
@AfterClass
public void tearDown()
{
    //Close browser
    d.quit();
}`script:


问:
How to extract substring from directory name using bash
I have a group of directories with the following name template:

[random combination of numbers and/or characters and/or symbols(including blank space)]+[single space]+[(]+[number]+[)].

E.g.
asdf%k123 test1 (12345)
foo(123)??bar (456)
324(asd)! (456)
random()123 (456)

How do I go about extracting only the number within parenthesis using bash commands (without '()')? Keep in mind that number within parenthesis might occur in the random part of name, but I only need the on at the end. In any case every name will end with [space][(][number][)].

答:
Would you please try the following;

```<code>pat='[[:blank:]]\(([[:digit:]]+)\)/$'   # regex pattern explained below
path="."                                # or specify to the path where the directories exist
for d in "$path"/*/; do                 # pick directories in the $path
    if [[ $d =~ $pat ]]; then           # if the directory name matches the pattern
        echo "${BASH_REMATCH[1]}"       # then print the extracted number
    fi
done
</code>```

Result with the provided examples:

```<code>456
12345
456
456
</code>```

The regex pattern [[:blank:]]\(([[:digit:]]+)\)/$ matches a substring such as:

a space [[:blank:]]

followed by a left paren \(

followed by a sequence of one or more digits ([[:digit:]]+)

followed by a right paren \)

followed by a slash (meaning it is a directory) /

and the end of string $

The sequence of digits is surrounded by parens so the matched
portion is captured and assigned to the bash variable BASH_REMATCH.

Body:
Would you please try the following;

pat='[[:blank:]]\(([[:digit:]]+)\)/$'   # regex pattern explained below
path="."                                # or specify to the path where the directories exist
for d in "$path"/*/; do                 # pick directories in the $path
    if [[ $d =~ $pat ]]; then           # if the directory name matches the pattern
        echo "${BASH_REMATCH[1]}"       # then print the extracted number
    fi
done

Result with the provided examples:

456
12345
456
456

The regex pattern [[:blank:]]\(([[:digit:]]+)\)/$ matches a substring such as:

a space [[:blank:]]

followed by a left paren \(

followed by a sequence of one or more digits ([[:digit:]]+)

followed by a right paren \)

followed by a slash (meaning it is a directory) /

and the end of string $

The sequence of digits is surrounded by parens so the matched
portion is captured and assigned to the bash variable BASH_REMATCH.


问:
How do I get formatted JSON in .NET using C#?
I am using .NET JSON parser and would like to serialize my config file so it is readable. So instead of:

{"blah":"v", "blah2":"v2"}
```
<p>I would like something nicer like:</p>
<pre class="lang-json prettyprint-override"><code>{
    "blah":"v",
    "blah2":"v2"
}
</code>```

My code is something like this:

using System.Web.Script.Serialization;

var ser = new JavaScriptSerializer();
configSz = ser.Serialize(config);
using (var f = (TextWriter)File.CreateText(configFn))
{
    f.WriteLine(configSz);
    f.Close();
}
```

答:
Here is a solution using Microsoft's System.Text.Json library:</p>

```<code>static string FormatJsonText(string jsonString)
{
    using var doc = JsonDocument.Parse(
        jsonString,
        new JsonDocumentOptions
        {
            AllowTrailingCommas = true
        }
    );
    MemoryStream memoryStream = new MemoryStream();
    using (
        var utf8JsonWriter = new Utf8JsonWriter(
            memoryStream,
            new JsonWriterOptions
            {
                Indented = true
            }
        )
    )
    {
        doc.WriteTo(utf8JsonWriter);
    }
    return new System.Text.UTF8Encoding()
        .GetString(memoryStream.ToArray());
}
</code>```

Body:
I am using .NET JSON parser and would like to serialize my config file so it is readable. So instead of:

{"blah":"v", "blah2":"v2"}

I would like something nicer like:

{
    "blah":"v",
    "blah2":"v2"
}

My code is something like this:

using System.Web.Script.Serialization;

var ser = new JavaScriptSerializer();
configSz = ser.Serialize(config);
using (var f = (TextWriter)File.CreateText(configFn))
{
    f.WriteLine(configSz);
    f.Close();
}

Body:
Here is a solution using Microsoft's System.Text.Json library:

static string FormatJsonText(string jsonString)
{
    using var doc = JsonDocument.Parse(
        jsonString,
        new JsonDocumentOptions
        {
            AllowTrailingCommas = true
        }
    );
    MemoryStream memoryStream = new MemoryStream();
    using (
        var utf8JsonWriter = new Utf8JsonWriter(
            memoryStream,
            new JsonWriterOptions
            {
                Indented = true
            }
        )
    )
    {
        doc.WriteTo(utf8JsonWriter);
    }
    return new System.Text.UTF8Encoding()
        .GetString(memoryStream.ToArray());
}


问:
Adding a list of security groups using Boto3 gives error "The security group does not exist"
I have a list of security groups that I want to add to some instances using the boto3 client modify_instance_attribute method.

Using the following code:

```<code>def attach_sg_list(ec2_client, sg_list, instance_id):
    sg_list = str(sg_list).replace(' ', '').replace('[','').replace(']','').replace('\'','')
    print(f"SG List: {sg_list}")
    try:
        attach_sg_response = ec2_client.modify_instance_attribute(
            InstanceId=instance_id,
            Groups=[
                sg_list,
            ]
        )
    except Exception as e:
        print(f"An error has occurred: {e}")
</code>```

I get the following output:

```<code>SG List: sg-0d0ddf3117d23cadb,sg-0e4b5fc1d40185fc3,sg-031ac185d029cd5fd,sg-0afa867f9029bb468,sg-2cad407c
An error has occurred: An error occurred (InvalidGroup.NotFound) when calling the ModifyInstanceAttribute operation: The security group 'sg-0d0ddf3117d23cadb,sg-0e4b5fc1d40185fc3,sg-031ac185d029cd5fd,sg-0afa867f9029bb468,sg-2cad407c' does not exist
</code>```

The description of Group for modify_instance_attribute is this:

```<code>Groups (list) --
[EC2-VPC] Changes the security groups of the instance. You must specify at least one security group, even if it's just the default security group for the VPC. You must specify the security group ID, not the security group name.

(string) --
</code>```

It says groups is a list and then says to specify a string. If I try giving it a list I get an error saying that it wants a string. This is the error I get if I do that:

```<code>Parameter validation failed:
Invalid type for parameter Groups[0], value: [' sg-031ac185d029cd5fd', ' sg-0d0ddf3117d23cadb', ' sg-05ef09508245e56bc', ' sg-0e4b5fc1d40185fc3', ' sg-2cad407c'], type: <class 'list'>, valid types: <class 'str'>
</code>```

It also says you can add 'at least one security group'.

How can I can assign a list of security group IDs to an ec2 instance using boto3?

答:
You did not show your code for the second error (giving a list), but it appears you are supplying a list as the first element of a list.

For example, if sg_list is a list of strings, it would appear you were using:

```<code>Groups = [sg_list]
</code>```

This would create a list of a list.

Instead, use:

```<code>Groups = sg_list
</code>```

Body:
I have a list of security groups that I want to add to some instances using the boto3 client modify_instance_attribute method.

Using the following code:

def attach_sg_list(ec2_client, sg_list, instance_id):
    sg_list = str(sg_list).replace(' ', '').replace('[','').replace(']','').replace('\'','')
    print(f"SG List: {sg_list}")
    try:
        attach_sg_response = ec2_client.modify_instance_attribute(
            InstanceId=instance_id,
            Groups=[
                sg_list,
            ]
        )
    except Exception as e:
        print(f"An error has occurred: {e}")

I get the following output:

SG List: sg-0d0ddf3117d23cadb,sg-0e4b5fc1d40185fc3,sg-031ac185d029cd5fd,sg-0afa867f9029bb468,sg-2cad407c
An error has occurred: An error occurred (InvalidGroup.NotFound) when calling the ModifyInstanceAttribute operation: The security group 'sg-0d0ddf3117d23cadb,sg-0e4b5fc1d40185fc3,sg-031ac185d029cd5fd,sg-0afa867f9029bb468,sg-2cad407c' does not exist

The description of Group for modify_instance_attribute is this:

Groups (list) --
[EC2-VPC] Changes the security groups of the instance. You must specify at least one security group, even if it's just the default security group for the VPC. You must specify the security group ID, not the security group name.

(string) --

It says groups is a list and then says to specify a string. If I try giving it a list I get an error saying that it wants a string. This is the error I get if I do that:

Parameter validation failed:
Invalid type for parameter Groups[0], value: [' sg-031ac185d029cd5fd', ' sg-0d0ddf3117d23cadb', ' sg-05ef09508245e56bc', ' sg-0e4b5fc1d40185fc3', ' sg-2cad407c'], type: <class 'list'>, valid types: <class 'str'>

It also says you can add 'at least one security group'.

How can I can assign a list of security group IDs to an ec2 instance using boto3?

Body:
You did not show your code for the second error (giving a list), but it appears you are supplying a list as the first element of a list.

For example, if sg_list is a list of strings, it would appear you were using:

Groups = [sg_list]

This would create a list of a list.

Instead, use:

Groups = sg_list


问:
Laravel 5.8 Email with SMTP and Office 365
I have been trying to send email for password reset but never receive the email.

No error on the page so not sure how to debug it.

Following is my .env configuration

```<code>MAIL_DRIVER=smtp
MAIL_HOST=smtp.mailtrap.io
MAIL_PORT=2525
MAIL_USERNAME=myUsername
MAIL_PASSWORD=myPassword
MAIL_FROM_ADDRESS='From email address'
MAIL_FROM_NAME='Demo App'
</code>```

Now when I reset the password, I get success message but do not receive email.

When I use following piece of code just to test emails, it works fine

```<code>\Mail::send('emails.send', ['title' => 'Password Rest', 'message' => 'Message has the token to reset password'], function ($message)
    {
        $message->from('from email address', 'Scotch.IO');
        $message->to('To email address');
    });
</code>```

I am just testing with mailtrap but will be using Office 365 with following configuration

```<code>MAIL_DRIVER=smtp
MAIL_HOST=smtp.office365.com
</code>```

How do I debug the mail error or resolve this problem?

Thank you

答:
When I use following piece of code just to test emails, it works fine

Since the Mail::send() works as intended then the mail configuration shouldn't be the issue.

Try to debug by putting some dd() before you return the success message and figure out if your password reset mail sending code is even triggered. Then switch to office 365 once you get mailtrap working.

EDIT after OP comments:

It seems like user class might not be set up properly. Please double check that the user class implements Illuminate\Notifications\Notifiable trait as per the warning in the documentation

If all else fail you can try overriding the sendPasswordResetNotification function as per the documentation under the reset email customization header. That function is the actual function being called by sendResetLinkEmail()

Body:
I have been trying to send email for password reset but never receive the email.

No error on the page so not sure how to debug it.

Following is my .env configuration

MAIL_DRIVER=smtp
MAIL_HOST=smtp.mailtrap.io
MAIL_PORT=2525
MAIL_USERNAME=myUsername
MAIL_PASSWORD=myPassword
MAIL_FROM_ADDRESS='From email address'
MAIL_FROM_NAME='Demo App'

Now when I reset the password, I get success message but do not receive email.

When I use following piece of code just to test emails, it works fine

\Mail::send('emails.send', ['title' => 'Password Rest', 'message' => 'Message has the token to reset password'], function ($message)
    {
        $message->from('from email address', 'Scotch.IO');
        $message->to('To email address');
    });

I am just testing with mailtrap but will be using Office 365 with following configuration

MAIL_DRIVER=smtp
MAIL_HOST=smtp.office365.com

How do I debug the mail error or resolve this problem?

Thank you


问:
Canvas in react not rerendering every props update
I have been trying to implement a scroll video parallax effect inspired by Apple's landing page
in React.js. I have gotten pretty close following these references:

https://codepen.io/ollieRogers/pen/lfeLc/

http://www.petecorey.com/blog/2019/08/19/animating-a-canvas-with-react-hooks/

However when scrolling, the frame only updates when I stop scrolling. My desired result for the canvas is to render in a fluid like fashion while scrolling. In other words the frame should be updating every time my props.scrollPos updates.

All that's important to know about the parent is that it's passing a scroll position using scrollTop. This component is suppose to be reusable and work relative to its parent which is why I chose to not keep a scroll state internally.

You can view a working example of my code here on CodeSandbox

ParallaxVideo.js

```<code>...

// request animation frame
// when scrollpos updates the current frame needs to change and cause rerender
// frame = (scrollPos - offset) / playbackConst

const ParallaxVideo = props => {
  ...

  useEffect(() => {
    let requestId;
    const context = canvasRef.current.getContext("2d")
    const offset = heightRef.current.offsetTop
    videoRef.current.currentTime = (props.scrollPos - offset) / playbackConst
    console.log(props.scrollPos)

    const render = () => {
      context.drawImage(videoRef.current, 0, 0)
      requestId = requestAnimationFrame(render)
    }

    render()

    return () => cancelAnimationFrame(requestId)
  })

  return (
    <div
      style={{height: scrollHeight}}
      ref={heightRef}
    >
      <VideoWrap>
        <Video
          muted
          preload={preload}
          autoPlay={autoPlay}
          loop={loop}
          ref={videoRef}
          fitToScreen={fitToScreen}
          onLoadedMetadata={updateHeight}
          playsInline
        >
          {props.children}
        </Video>
        <Canvas ref={canvasRef}/>
      </VideoWrap>
    </div>
  )
}
</code>```

答:
for sure will not change the useEffect should have a inputs set to listen for changes,</p>

```<code>useEffect(() => {
    let requestId;
    const context = canvasRef.current.getContext("2d")
    const offset = heightRef.current.offsetTop
    videoRef.current.currentTime = (props.scrollPos - offset) / playbackConst
    console.log(props.scrollPos)

    const render = () => {
      context.drawImage(videoRef.current, 0, 0)
      requestId = requestAnimationFrame(render)
    }

    render()

    return () => cancelAnimationFrame(requestId)
  },[props.scrollPos])
</code>```

Body:
I have been trying to implement a scroll video parallax effect inspired by Apple's landing page
in React.js. I have gotten pretty close following these references:

https://codepen.io/ollieRogers/pen/lfeLc/

http://www.petecorey.com/blog/2019/08/19/animating-a-canvas-with-react-hooks/

However when scrolling, the frame only updates when I stop scrolling. My desired result for the canvas is to render in a fluid like fashion while scrolling. In other words the frame should be updating every time my props.scrollPos updates.

All that's important to know about the parent is that it's passing a scroll position using scrollTop. This component is suppose to be reusable and work relative to its parent which is why I chose to not keep a scroll state internally.

You can view a working example of my code here on CodeSandbox

ParallaxVideo.js

...

// request animation frame
// when scrollpos updates the current frame needs to change and cause rerender
// frame = (scrollPos - offset) / playbackConst

const ParallaxVideo = props => {
  ...

  useEffect(() => {
    let requestId;
    const context = canvasRef.current.getContext("2d")
    const offset = heightRef.current.offsetTop
    videoRef.current.currentTime = (props.scrollPos - offset) / playbackConst
    console.log(props.scrollPos)

    const render = () => {
      context.drawImage(videoRef.current, 0, 0)
      requestId = requestAnimationFrame(render)
    }

    render()

    return () => cancelAnimationFrame(requestId)
  })

  return (

      <VideoWrap>
        <Video
          muted
          preload={preload}
          autoPlay={autoPlay}
          loop={loop}
          ref={videoRef}
          fitToScreen={fitToScreen}
          onLoadedMetadata={updateHeight}
          playsInline
        >
          {props.children}
        </Video>
        <Canvas ref={canvasRef}/>
      </VideoWrap>

  )
}

Body:
for sure will not change the useEffect should have a inputs set to listen for changes,

useEffect(() => {
    let requestId;
    const context = canvasRef.current.getContext("2d")
    const offset = heightRef.current.offsetTop
    videoRef.current.currentTime = (props.scrollPos - offset) / playbackConst
    console.log(props.scrollPos)

    const render = () => {
      context.drawImage(videoRef.current, 0, 0)
      requestId = requestAnimationFrame(render)
    }

    render()

    return () => cancelAnimationFrame(requestId)
  },[props.scrollPos])


问:
How to send a GraphQL query to AppSync from python?
How do we post a GraphQL request through AWS AppSync using boto?

Ultimately I'm trying to mimic a mobile app accessing our stackless/cloudformation stack on AWS, but with python. Not javascript or amplify.

The primary pain point is authentication; I've tried a dozen different ways already.  This the current one, which generates a "401" response with "UnauthorizedException" and "Permission denied", which is actually pretty good considering some of the other messages I've had.  I'm now using the 'aws_requests_auth' library to do the signing part.  I assume it authenticates me using the stored /.aws/credentials from my local environment, or does it?

I'm a little confused as to where and how cognito identities and pools will come into it. eg: say I wanted to mimic the sign-up sequence?

Anyways the code looks pretty straightforward; I just don't grok the authentication.

```<code>from aws_requests_auth.boto_utils import BotoAWSRequestsAuth

APPSYNC_API_KEY = 'inAppsyncSettings'
APPSYNC_API_ENDPOINT_URL = 'https://aaaaaaaaaaaavzbke.appsync-api.ap-southeast-2.amazonaws.com/graphql'

headers = {
    'Content-Type': "application/graphql",
    'x-api-key': APPSYNC_API_KEY,
    'cache-control': "no-cache",
}
query = """{
    GetUserSettingsByEmail(email: "john@washere"){
      items {name, identity_id, invite_code}
    }
}"""

def test_stuff():
    # Use the library to generate auth headers.
    auth = BotoAWSRequestsAuth(
        aws_host='aaaaaaaaaaaavzbke.appsync-api.ap-southeast-2.amazonaws.com',
        aws_region='ap-southeast-2',
        aws_service='appsync')

    # Create an http graphql request.
    response = requests.post(
        APPSYNC_API_ENDPOINT_URL,
        json={'query': query},
        auth=auth,
        headers=headers)

    print(response)

# this didn't work:
#    response = requests.post(APPSYNC_API_ENDPOINT_URL, data=json.dumps({'query': query}), auth=auth, headers=headers)
</code>```

Yields

```<code>{
  "errors" : [ {
    "errorType" : "UnauthorizedException",
    "message" : "Permission denied"
  } ]
}
</code>```

答:
It's quite simple--once you know. There are some things I didn't appreciate:

I've assumed IAM authentication (OpenID appended way below)

There are a number of ways for appsync to handle authentication. We're using IAM so that's what I need to deal with, yours might be different.

Boto doesn't come into it.

We want to issue a request like any regular punter, they don't use boto, and neither do we.  Trawling the AWS boto docs was a waste of time.

Use the AWS4Auth library
We are going to send a regular http request to aws, so whilst we can use python requests they need to be authenticated--by attaching headers.
And, of course, AWS auth headers are special and different from all others.
You can try to work out how to do it
yourself, or you can go looking for someone else who has already done it: Aws_requests_auth, the one I started with, probably works just fine, but I have ended up with AWS4Auth.  There are many others of dubious value; none endorsed or provided by Amazon (that I could find).

Specify appsync as the "service"

What service are we calling?  I didn't find any examples of anyone doing this anywhere.  All the examples are trivial S3 or EC2 or even EB which left uncertainty. Should we be talking to api-gateway service?  Whatsmore, you feed this detail into the AWS4Auth routine, or authentication data.  Obviously, in hindsight, the request is hitting Appsync, so it will be authenticated by Appsync, so specify "appsync" as the service when putting together the auth headers.

It comes together as:

```<code>import requests
from requests_aws4auth import AWS4Auth

# Use AWS4Auth to sign a requests session
session = requests.Session()
session.auth = AWS4Auth(
    # An AWS 'ACCESS KEY' associated with an IAM user.
    'AKxxxxxxxxxxxxxxx2A',
    # The 'secret' that goes with the above access key.
    'kwWxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxgEm',
    # The region you want to access.
    'ap-southeast-2',
    # The service you want to access.
    'appsync'
)
# As found in AWS Appsync under Settings for your endpoint.
APPSYNC_API_ENDPOINT_URL = 'https://nqxxxxxxxxxxxxxxxxxxxke'
                           '.appsync-api.ap-southeast-2.amazonaws.com/graphql'
# Use JSON format string for the query. It does not need reformatting.
query = """
    query foo {
        GetUserSettings (
           identity_id: "ap-southeast-2:8xxxxxxb-7xx4-4xx4-8xx0-exxxxxxx2"
        ){
           user_name, email, whatever
}}"""
# Now we can simply post the request...
response = session.request(
    url=APPSYNC_API_ENDPOINT_URL,
    method='POST',
    json={'query': query}
)
print(response.text)

</code>```

Which yields

```<code># Your answer comes as a JSON formatted string in the text attribute, under data.
{"data":{"GetUserSettings":{"user_name":"0xxxxxxx3-9102-42f0-9874-1xxxxx7dxxx5"}}}
</code>```

Getting credentials

To get rid of the hardcoded key/secret you can consume the local AWS ~/.aws/config and ~/.aws/credentials, and it is done this way...

```<code># Use AWS4Auth to sign a requests session
session = requests.Session()
credentials = boto3.session.Session().get_credentials()
session.auth = AWS4Auth(
    credentials.access_key,
    credentials.secret_key,
    boto3.session.Session().region_name,
    'appsync',
    session_token=credentials.token
)
...<as above>
</code>```

This does seem to respect the environment variable AWS_PROFILE for assuming different roles.

Note that STS.get_session_token is not the way to do it, as it may try to assume a role from a role, depending where it keyword matched the AWS_PROFILE value.  Labels in the credentials file will work because the keys are right there, but names found in the config file do not work, as that assumes a role already.

OpenID

In this scenario, all the complexity is transferred to the conversation with the openid connect provider.  The hard stuff is all the auth hoops you jump through to get an access token, and thence using the refresh token to keep it alive.  That is where all the real work lies.

Once you finally have an access token, assuming you have configured the "OpenID Connect" Authorization Mode in appsync, then you can, very simply, drop the access token into the header:

```<code>response = requests.post(
    url="https://nc3xxxxxxxxxx123456zwjka.appsync-api.ap-southeast-2.amazonaws.com/graphql",
    headers={"Authorization": ACCESS_TOKEN},
    json={'query': "query foo{GetStuff{cat, dog, tree}}"}
)
</code>```

Body:
How do we post a GraphQL request through AWS AppSync using boto?

Ultimately I'm trying to mimic a mobile app accessing our stackless/cloudformation stack on AWS, but with python. Not javascript or amplify.

The primary pain point is authentication; I've tried a dozen different ways already.  This the current one, which generates a "401" response with "UnauthorizedException" and "Permission denied", which is actually pretty good considering some of the other messages I've had.  I'm now using the 'aws_requests_auth' library to do the signing part.  I assume it authenticates me using the stored /.aws/credentials from my local environment, or does it?

I'm a little confused as to where and how cognito identities and pools will come into it. eg: say I wanted to mimic the sign-up sequence?

Anyways the code looks pretty straightforward; I just don't grok the authentication.

from aws_requests_auth.boto_utils import BotoAWSRequestsAuth

APPSYNC_API_KEY = 'inAppsyncSettings'
APPSYNC_API_ENDPOINT_URL = 'https://aaaaaaaaaaaavzbke.appsync-api.ap-southeast-2.amazonaws.com/graphql'

headers = {
    'Content-Type': "application/graphql",
    'x-api-key': APPSYNC_API_KEY,
    'cache-control': "no-cache",
}
query = """{
    GetUserSettingsByEmail(email: "john@washere"){
      items {name, identity_id, invite_code}
    }
}"""

def test_stuff():
    # Use the library to generate auth headers.
    auth = BotoAWSRequestsAuth(
        aws_host='aaaaaaaaaaaavzbke.appsync-api.ap-southeast-2.amazonaws.com',
        aws_region='ap-southeast-2',
        aws_service='appsync')

    # Create an http graphql request.
    response = requests.post(
        APPSYNC_API_ENDPOINT_URL,
        json={'query': query},
        auth=auth,
        headers=headers)

    print(response)

# this didn't work:
#    response = requests.post(APPSYNC_API_ENDPOINT_URL, data=json.dumps({'query': query}), auth=auth, headers=headers)

Yields

{
  "errors" : [ {
    "errorType" : "UnauthorizedException",
    "message" : "Permission denied"
  } ]
}

Body:
It's quite simple--once you know. There are some things I didn't appreciate:

I've assumed IAM authentication (OpenID appended way below)

There are a number of ways for appsync to handle authentication. We're using IAM so that's what I need to deal with, yours might be different.

Boto doesn't come into it.

We want to issue a request like any regular punter, they don't use boto, and neither do we.  Trawling the AWS boto docs was a waste of time.

Use the AWS4Auth library
We are going to send a regular http request to aws, so whilst we can use python requests they need to be authenticated--by attaching headers.
And, of course, AWS auth headers are special and different from all others.
You can try to work out how to do it
yourself, or you can go looking for someone else who has already done it: Aws_requests_auth, the one I started with, probably works just fine, but I have ended up with AWS4Auth.  There are many others of dubious value; none endorsed or provided by Amazon (that I could find).

Specify appsync as the "service"

What service are we calling?  I didn't find any examples of anyone doing this anywhere.  All the examples are trivial S3 or EC2 or even EB which left uncertainty. Should we be talking to api-gateway service?  Whatsmore, you feed this detail into the AWS4Auth routine, or authentication data.  Obviously, in hindsight, the request is hitting Appsync, so it will be authenticated by Appsync, so specify "appsync" as the service when putting together the auth headers.

It comes together as:

import requests
from requests_aws4auth import AWS4Auth

# Use AWS4Auth to sign a requests session
session = requests.Session()
session.auth = AWS4Auth(
    # An AWS 'ACCESS KEY' associated with an IAM user.
    'AKxxxxxxxxxxxxxxx2A',
    # The 'secret' that goes with the above access key.
    'kwWxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxgEm',
    # The region you want to access.
    'ap-southeast-2',
    # The service you want to access.
    'appsync'
)
# As found in AWS Appsync under Settings for your endpoint.
APPSYNC_API_ENDPOINT_URL = 'https://nqxxxxxxxxxxxxxxxxxxxke'
                           '.appsync-api.ap-southeast-2.amazonaws.com/graphql'
# Use JSON format string for the query. It does not need reformatting.
query = """
    query foo {
        GetUserSettings (
           identity_id: "ap-southeast-2:8xxxxxxb-7xx4-4xx4-8xx0-exxxxxxx2"
        ){
           user_name, email, whatever
}}"""
# Now we can simply post the request...
response = session.request(
    url=APPSYNC_API_ENDPOINT_URL,
    method='POST',
    json={'query': query}
)
print(response.text)

Which yields

# Your answer comes as a JSON formatted string in the text attribute, under data.
{"data":{"GetUserSettings":{"user_name":"0xxxxxxx3-9102-42f0-9874-1xxxxx7dxxx5"}}}

Getting credentials

To get rid of the hardcoded key/secret you can consume the local AWS ~/.aws/config and ~/.aws/credentials, and it is done this way...

# Use AWS4Auth to sign a requests session
session = requests.Session()
credentials = boto3.session.Session().get_credentials()
session.auth = AWS4Auth(
    credentials.access_key,
    credentials.secret_key,
    boto3.session.Session().region_name,
    'appsync',
    session_token=credentials.token
)
...<as above>

This does seem to respect the environment variable AWS_PROFILE for assuming different roles.

Note that STS.get_session_token is not the way to do it, as it may try to assume a role from a role, depending where it keyword matched the AWS_PROFILE value.  Labels in the credentials file will work because the keys are right there, but names found in the config file do not work, as that assumes a role already.

OpenID

In this scenario, all the complexity is transferred to the conversation with the openid connect provider.  The hard stuff is all the auth hoops you jump through to get an access token, and thence using the refresh token to keep it alive.  That is where all the real work lies.

Once you finally have an access token, assuming you have configured the "OpenID Connect" Authorization Mode in appsync, then you can, very simply, drop the access token into the header:

response = requests.post(
    url="https://nc3xxxxxxxxxx123456zwjka.appsync-api.ap-southeast-2.amazonaws.com/graphql",
    headers={"Authorization": ACCESS_TOKEN},
    json={'query': "query foo{GetStuff{cat, dog, tree}}"}
)


问:
Prometheus - How to reuse a filtered series to speed up query time?
Now I'm using Prometheus as Time series DB. I want to reuse a filtered series to optimize the query time.

Now I have a time series command_stat{id="x",host="x.x.x.x"}, example:

```<code>command_stat{id="0",host="10.x.x.1",stat="requests"}
command_stat{id="0",host="10.x.x.2",stat="requests"}
command_stat{id="1",host="10.x.x.1",stat="requests"}
command_stat{id="1",host="10.x.x.2",stat="requests"}
command_stat{id="2",host="10.x.x.1",stat="requests"}
command_stat{id="2",host="10.x.x.2",stat="requests"}
...
</code>```

I have about 10000 commands and about 500 command id

I'll calculate the total of rate on each command id, which is:

```<code>sum(rate(command_stat{stat="requests"}[5m])) by (id)
</code>```

And I should compare this value with values in the last 15 days. Example, I'll calculate:

```<code>sum(rate(command_stat{stat="requests"}[5m] offset 1d)) by (id)
sum(rate(command_stat{stat="requests"}[5m] offset 2d)) by (id)
sum(rate(command_stat{stat="requests"}[5m] offset 3d)) by (id)
...
sum(rate(command_stat{stat="requests"}[5m] offset 15d)) by (id)
</code>```

I should also calculate mean and standard deviation from 15 values.
With mean, I can use avg(...) by (id) and for standard deviation, I can use stddev(...) by (id)

Example for standard deviation, that should be:

```<code>stddev (
  label_replace(sum(rate(command_stat{stat_name="requests"}[5m] offset 1d)) by (id), "time", "1d", "", ".*") or
  label_replace(sum(rate(command_stat{stat_name="requests"}[5m] offset 2d)) by (id), "time", "2d", "", ".*") or
  ...
  label_replace(sum(rate(command_stat{stat_name="requests"}[5m] offset 15d)) by (id), "time", "15d", "", ".*")
) by (id)
</code>```

The query time is too slow, I want to speed up by checking only command id with the condition:

```<code>sum(rate(command_stat{stat="requests"}[5m])) by (id) > 1000
</code>```

After this condition, the number of commands that should be checked is only about 40. I want to calculate mean and standard deviation only on the ~40 commands.

Can you help me optimize this problem? I think we can reuse the command_stat series after filter by above condition.

答:
I've missed recording rules in Prometheus. I think I can use this feature to pre-compute the series:

```<code>sum(rate(command_stat{stat="requests"}[5m])) by (id)
</code>```

And then I can calculate mean and standard deviation.

If you have a better solution, please let me know to improve.

Thanks!!

Body:
Now I'm using Prometheus as Time series DB. I want to reuse a filtered series to optimize the query time.

Now I have a time series command_stat{id="x",host="x.x.x.x"}, example:

command_stat{id="0",host="10.x.x.1",stat="requests"}
command_stat{id="0",host="10.x.x.2",stat="requests"}
command_stat{id="1",host="10.x.x.1",stat="requests"}
command_stat{id="1",host="10.x.x.2",stat="requests"}
command_stat{id="2",host="10.x.x.1",stat="requests"}
command_stat{id="2",host="10.x.x.2",stat="requests"}
...

I have about 10000 commands and about 500 command id

I'll calculate the total of rate on each command id, which is:

sum(rate(command_stat{stat="requests"}[5m])) by (id)

And I should compare this value with values in the last 15 days. Example, I'll calculate:

sum(rate(command_stat{stat="requests"}[5m] offset 1d)) by (id)
sum(rate(command_stat{stat="requests"}[5m] offset 2d)) by (id)
sum(rate(command_stat{stat="requests"}[5m] offset 3d)) by (id)
...
sum(rate(command_stat{stat="requests"}[5m] offset 15d)) by (id)

I should also calculate mean and standard deviation from 15 values.
With mean, I can use avg(...) by (id) and for standard deviation, I can use stddev(...) by (id)

Example for standard deviation, that should be:

stddev (
  label_replace(sum(rate(command_stat{stat_name="requests"}[5m] offset 1d)) by (id), "time", "1d", "", ".*") or
  label_replace(sum(rate(command_stat{stat_name="requests"}[5m] offset 2d)) by (id), "time", "2d", "", ".*") or
  ...
  label_replace(sum(rate(command_stat{stat_name="requests"}[5m] offset 15d)) by (id), "time", "15d", "", ".*")
) by (id)

The query time is too slow, I want to speed up by checking only command id with the condition:

sum(rate(command_stat{stat="requests"}[5m])) by (id) > 1000

After this condition, the number of commands that should be checked is only about 40. I want to calculate mean and standard deviation only on the ~40 commands.

Can you help me optimize this problem? I think we can reuse the command_stat series after filter by above condition.

Body:
I've missed recording rules in Prometheus. I think I can use this feature to pre-compute the series:

sum(rate(command_stat{stat="requests"}[5m])) by (id)

And then I can calculate mean and standard deviation.

If you have a better solution, please let me know to improve.

Thanks!!


问:
Android transition animation API 19 and above
I'm quite new to animation transition. I want to add an animation in the home screen. I don't know which way to go, is Lottie fit for this or official android libraries.
The animation take about 2-3 seconds and should animate every time I go to that specific screen. Please help, some explanations would be great Here's some screenshots:

答:
You can use ObjectAnimator and AnimatorSet.
Translate that blocks to bottom in order and with some delay.
The code you need is something like that:

```<code>ObjectAnimator translateAnimator1 = ObjectAnimator.ofFloat(view1, "translationY", 0, 500).setDuration(2500);
ObjectAnimator translateAnimator2 = ObjectAnimator.ofFloat(view2, "translationY", 0, 500).setDuration(2500);
translateAnimator2.setStartDelay(500);
ObjectAnimator translateAnimator3 = ObjectAnimator.ofFloat(view3, "translationY", 0, 500).setDuration(2500);
translateAnimator3.setStartDelay(1000);

AnimatorSet animatorSet = new AnimatorSet();
animatorSet.play(translateAnimator1).with(translateAnimator2).with(translateAnimator3);
animatorSet.start();
</code>```

ObjectAnimator and AnimatorSet has some extra properties like setRepeatCount etc. that maybe useful.

Body:
You can use ObjectAnimator and AnimatorSet.
Translate that blocks to bottom in order and with some delay.
The code you need is something like that:

ObjectAnimator translateAnimator1 = ObjectAnimator.ofFloat(view1, "translationY", 0, 500).setDuration(2500);
ObjectAnimator translateAnimator2 = ObjectAnimator.ofFloat(view2, "translationY", 0, 500).setDuration(2500);
translateAnimator2.setStartDelay(500);
ObjectAnimator translateAnimator3 = ObjectAnimator.ofFloat(view3, "translationY", 0, 500).setDuration(2500);
translateAnimator3.setStartDelay(1000);

AnimatorSet animatorSet = new AnimatorSet();
animatorSet.play(translateAnimator1).with(translateAnimator2).with(translateAnimator3);
animatorSet.start();

ObjectAnimator and AnimatorSet has some extra properties like setRepeatCount etc. that maybe useful.