问:
R: Append data when points overlap/within distance; add buffer rectangle to set1, add radius to set2
I have a database (netcdf) of eddies moving through the ocean, and a number of tracks of fish doing the same. I'm trying to append eddy info to the fish track when the fish is within range of the eddy (locations within a (varying) threshold distance and dates identical). Data structure, complications, and reproducible example follow. I suspect this isn't super hard, I just keep tying myself in knots trying to work out a neat solution. Thanks in advance for any thoughts!

Data structure / reproducible example: 3 same-days of fish & eddy track

```<code>library(lubridate)
fish <- data.frame(lat = c(42.1, 42.6, 43.2), lon = c(-10, -10.1, -10.2), date = ymd(c("1990-01-01", "1990-01-02", "1990-01-02")))
eddy <- data.frame(lat = c(44, 42.3, 40), lon = c(-15, -10.1, -6), date = ymd(c("1990-01-01", "1990-01-02", "1990-01-02")), track = c(1,1,1), radius = c(81,82,83), vals = c(11,12,13))
</code>```

Here's where it gets a bit trickier:

Varying radius circular eddy buffer

The size of the eddy (from which to calculate overlap or distance) is given by the radius column, so:

```<code>sf_eddy <- st_as_sf(eddy)
st_buffer(sf_eddy, dist = radius)
</code>```

For st_buffer "All operations work on a per-feature basis" so I'm not sure if that'll treat sf_eddy as a single feature (probably (multi)linestring) or nrows of separate points. I imagine I can play around with that until I get it to work, such that each eddy-day (n=151937) is an sf buffer circle which I can presumably append its vals to, if they're not included already.

Set-distance latlong rectangle fish buffer

There's a fixed error distance for fish positions of +-1.9°lat, +-0.77°lon. So conceptually I want to create a tall thin rectangle polygon around each day's fish location. st_buffer doesn't look to allow this but there may be other options, potentially by creating a polygon around each centroid as per the square example here.

If fish & any eddy overlap in space & time, append eddy vals to fish

I could potentially slash the workload by only running the above steps when eddy$date==fish$date? Though the eddy calculation would be a one-off and I can save the output. I suspect the fish box calculation won't be particularly taxing either. Anyway:

```<code>fish$vals <- rep(NA, nrow(fish))
sf_fish <- sf_as_sf(fish)
for (i in seq(along = sf_fish)){
if(st_intersects(sf_fish[i,],sf_eddy)) sf_fish[i,"vals"] <- sf_eddy$vals}
</code>```

This is probably an awful way of doing it, obviously.

But is any of this even vaguely a reasonable way of solving this problem, or am I missing much more elegant solutions due to my amateurish knowledge of sf, sp, rgeos, etc? I have 166 fish files with average 286 days of locations = 47476 total lookups. Would it be good/fast to create an index of fish/eddy date match pairs (there may be multiple eddies on the same day) and then only test intersections on those? Since I need to append eddy$vals (actually 2 or 3 columns) to fish, would it be better to use spatial join, perhaps st_join?

Edit: comment & output map oddities:
600,000 total fish/eddy row ggplot output:

Looks like I need to fix the projection as the eddies aren't printing west of 0. They're definitely in the same CRS now; maybe the originals are 0-360 not -180:180. Also while fish boxes print nicely, eddies are too small: mean r=80km=~1deg@45N, and they're definitely printing much smaller than that. I'll have a think about that too. Also: will for() have a problem if 2 eddies exist on the same day?

Edit2: originals were 0:360 which I corrected. Eddy radii still too small:

Per code discussion, original eddy file with latlons was converted with st_as_sf to crs4326 (wgs84) then st_transform(sf_eddy,6931) which is North Atlantic, per image above. geometry column in resultant file is POLYGON, XY dimension, values are in projected coordinates not latlons, so e.g. sf_eddy_buffered$geometry[1] bbox: xmin -4894790 ymin -2418555 xmax -4894648 ymax -2418413. speed_radius is 71 so the difference between xmin & xmax is 142 which is 71*2 which is right. Does radius unit not equate to projection distance unit? Stewart's examples were 81000. Eddy radius unit: km. Projection / geometry unit: https://epsg.io/6931 says units are indeed metre. Multiply radius by 1000 & try again:

So that's all good. Just tried the updated st_join code, think it could use some work - takes absolutely forever probably on account of the 3639209 eddy rows vs 461 fish rows. Also it has created multiple rows for multiple eddies for the same fish on the same day. Potentially a function of the relatively large overlap opportunity given the error buffer box around the fish.
I suspect I'll be able to fix this by marking duplicates and deleting the duplicates which are furthest away. The current way of doing (via v and rbind) it is better than the previous way (direct cell assignment of a value) IMO since otherwise multiple values would be sent to a single cell, either crashing it or silently overwriting.

For speeding up the code I was thinking that I could subset the eddy file to only the overlap in date range of the current fish, and use that subset in the join loop. The join loop already subsets the fish and the eddy files by date anyway so maybe this is redundant, or maybe reducing the size of the eddy data that needs to be worked on will give the CPU/RAM a reduced workload each time?

Edit3: this DRASTICALLY speeds things up. Adding a simple loop counter for thisDate showed it completed at 275 of 461 rows. And produced a 481 length file. Possibly there are lots of days where the fish wasn't in an eddy (thisDate loop results in no v and no rbind), and lots of days of duplicates where the fish was in 2+ eddies (thisDate loop results in multirow v and rbind). Still feels like it should process 461 loops, however...

Edit4: a small tweak:

```<code>v <- st_join(thisFishRow, thisEddyRow, left = F, largest = TRUE)
</code>```

largest means it'll only join the eddy with the largest overlap area, resulting in a max v rowlength of 1 and thus 1 value per day max. However, nuances of the paths mean the fish can seem to go back and forth between different ongoing eddies. My suspicion (based on a similar study) is that they are more likely to stay within a strong anticyclonic eddy system. Radius (and thus area) of eddy controls overlap possibly more than position. For a 1.5 month section I found the highest-overlap eddy track number bounced between 3 eddies. Eddy track number 217008 is the most powerfully anticyclonic. Here's a (crap) plot:

As you can see, 217008 is right in the centre of the fish error box whereas the larger, more diffuse, lower powered eddies 217343 and 39625 are off to the edge. However their larger size often sees them bumped to the top because they have more area to give, and centroid proximity isn't counted.
So: if fishbox overlaps with eddy on same day then include eddy in shortlist
(thisFishRow & thisEddyRow stay the same). Then: Choose eddy from shortlist based on lowest distance from centroids (something instead of st_join). To be continued!

Edit5:

```<code>fishNearEddies <- NULL
for (thisDate in sf_df_nona$date) {
 thisFishRow <- sf_df_nona[sf_df_nona$date == thisDate, ]
 thisEddyRow <- sebDateSub[sebDateSub$date == thisDate, ]
 overlapToday <- st_join(thisFishRow, thisEddyRow, left = F)
 if (nrow(overlapToday) > 0) {
  overlapEddies <- thisEddyRow[thisEddyRow$track %in% overlapToday$track,]
  st_distance(x = thisFishRow, y = overlapEddies, by_element = TRUE) #0 0 due to overlap min dist = 0
 } #close if
} #close for
</code>```

st_distance won't work because the spatial features (buffer box & buffer circle) overlap, so min distance = 0. I ALSO need centroids to test against, I presume?

Edit6: Final Edit with working code, & answer award to Stewart for all his help. Thanks again sir.

```<code># overlap join loop####
fishNearEddies <- NULL
EdCentroidDist <- NULL
counter <- 1
ofhowmany <- length(sf_df_nona$date)
for (thisDate in sf_df_nona$date) {
  thisFishRow <- sf_df_nona[sf_df_nona$date == thisDate, ] #will be 1 row per day
  thisEddyRow <- sebDateSub[sebDateSub$date == thisDate, ] #all eddies that day, could be multi rows
  overlapToday <- st_join(thisFishRow, thisEddyRow, left = F) #will join only if they spatial intersect and (already) time match
  if (nrow(overlapToday) > 0) {
    # now need to join based on closest centroid and NOT on highest overlap
    overlapEddies <- thisEddyRow[thisEddyRow$track %in% overlapToday$track,] #subset TER by overlap tracks
    fishcentroid <- st_centroid(thisFishRow)
    eddycentroid <- st_centroid(overlapEddies)
    fishEdDists <- st_distance(x = fishcentroid, y = eddycentroid, by_element = TRUE) #result vector corresponding to overlapEddies rownumbers
    fishNearEddies <- rbind(fishNearEddies, overlapEddies[which.min(fishEdDists),]) #row index for overlapEddies, no index but has date
    EdCentroidDist <- rbind(EdCentroidDist, as.numeric(min(fishEdDists)))
  } #close if
  print(paste0(counter, " of ", ofhowmany, " fish days"))
  counter <- counter + 1
} #close for

if (!is.null(fishNearEddies)) {  #if there are overlaps, do processing. If not it'll fail

  fishNearEddies %<>% as.data.frame() %>% # convert to nonspatial so I can remove buffer column
    select(-geometry) %>% # remove geometry column. Attributes still remain. Whatever?
    cbind(EdCentroidDist) #add to FNE as column
</code>```

答:
Here's something that should get you going:

```<code>library(lubridate)
library(sf)
library(ggplot2)

# sample data
fish <- data.frame(
    lat = c(41.1, 43.6, 44.2),
    lon = c(-7, -11, -15),
    date = ymd(c("1990-01-01", "1990-01-02", "1990-01-03"))
)

# Add a blank column for the eddy values we want
fish$vals <- rep(NA, nrow(fish))

eddy <- data.frame(
    lat = c(44, 42.3, 40),
    lon = c(-6, -10.1, -15),
    date = ymd(c("1990-01-01", "1990-01-02", "1990-01-03")),
    track = c(1,1,1),
    radius = c(81000,82000,83000),
    vals = c(11,12,13)
)

# Convert eddy to simple features, using WGS84 CRS
sf_eddy <- st_as_sf(eddy,  coords = c("lon", "lat"), crs=4326)

# Transform from geographical to projected so we can buffer correctly. You might need to pick a different CRS
sf_eddy <- st_transform(sf_eddy, 3035) # ETRS89 / LAEA Europe

# Buffer the eddy points based on their radii
sf_eddy_buffered <- st_buffer(sf_eddy, dist = sf_eddy$radius)

# Add error to fish position. There's probably a better way to do this.
fishErrLat <- 1.9
fishErrLon <- 0.77
fish$buffer <- paste('POLYGON((',
                    fish$lon - fishErrLon, ' ', fish$lat + fishErrLat, ',',
                    fish$lon + fishErrLon, ' ', fish$lat + fishErrLat, ',',
                    fish$lon + fishErrLon, ' ', fish$lat - fishErrLat, ',',
                    fish$lon - fishErrLon, ' ', fish$lat - fishErrLat, ',',

                    fish$lon - fishErrLon, ' ', fish$lat + fishErrLat,
                    '))',
                    sep=''
                )

# Convert fish to simple features, using WGS84 CRS
sf_fish <- st_as_sf(fish, wkt='buffer', crs=4326)

# Transform from geographical to projected
sf_fish <- st_transform(sf_fish, 3035)

#Plot what we've got so far
g <- ggplot() +
        geom_sf(data=sf_eddy_buffered, aes(fill=date)) +
        geom_sf(data=sf_fish, aes(fill=date))

print(g)
</code>```

```<code># Check for overlap
fishNearEddies <- NULL

for (thisDate in unique(sf_fish$date)) {

    thisFishRow <- sf_fish[sf_fish$date==thisDate, ]
    thisEddyRow <- sf_eddy_buffered[sf_eddy_buffered$date==thisDate, ]

    v <- st_join(thisFishRow, thisEddyRow, left=F)
    if (nrow(v) > 0) {
        fishNearEddies <- rbind(fishNearEddies, v)
    }

}
</code>```

And check the results:

```<code>> fishNearEddies
Simple feature collection with 1 feature and 7 fields
geometry type:  POLYGON
dimension:      XY
bbox:           xmin: 2588691 ymin: 2402830 xmax: 2703342 ymax: 2624501
epsg (SRID):    3035
proj4string:    +proj=laea +lat_0=52 +lon_0=10 +x_0=4321000 +y_0=3210000 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs
   lat lon     date.x     date.y track radius vals                         buffer
2 43.6 -11 1990-01-02 1990-01-02     1  82000   12 POLYGON ((2644792 2624501, ...
</code>```

This will give you only those fish that intersect with an eddy.

Body:
I have a database (netcdf) of eddies moving through the ocean, and a number of tracks of fish doing the same. I'm trying to append eddy info to the fish track when the fish is within range of the eddy (locations within a (varying) threshold distance and dates identical). Data structure, complications, and reproducible example follow. I suspect this isn't super hard, I just keep tying myself in knots trying to work out a neat solution. Thanks in advance for any thoughts!

Data structure / reproducible example: 3 same-days of fish & eddy track

library(lubridate)
fish <- data.frame(lat = c(42.1, 42.6, 43.2), lon = c(-10, -10.1, -10.2), date = ymd(c("1990-01-01", "1990-01-02", "1990-01-02")))
eddy <- data.frame(lat = c(44, 42.3, 40), lon = c(-15, -10.1, -6), date = ymd(c("1990-01-01", "1990-01-02", "1990-01-02")), track = c(1,1,1), radius = c(81,82,83), vals = c(11,12,13))

Here's where it gets a bit trickier:

Varying radius circular eddy buffer

The size of the eddy (from which to calculate overlap or distance) is given by the radius column, so:

sf_eddy <- st_as_sf(eddy)
st_buffer(sf_eddy, dist = radius)

For st_buffer "All operations work on a per-feature basis" so I'm not sure if that'll treat sf_eddy as a single feature (probably (multi)linestring) or nrows of separate points. I imagine I can play around with that until I get it to work, such that each eddy-day (n=151937) is an sf buffer circle which I can presumably append its vals to, if they're not included already.

Set-distance latlong rectangle fish buffer

There's a fixed error distance for fish positions of +-1.9°lat, +-0.77°lon. So conceptually I want to create a tall thin rectangle polygon around each day's fish location. st_buffer doesn't look to allow this but there may be other options, potentially by creating a polygon around each centroid as per the square example here.

If fish & any eddy overlap in space & time, append eddy vals to fish

I could potentially slash the workload by only running the above steps when eddy$date==fish$date? Though the eddy calculation would be a one-off and I can save the output. I suspect the fish box calculation won't be particularly taxing either. Anyway:

fish$vals <- rep(NA, nrow(fish))
sf_fish <- sf_as_sf(fish)
for (i in seq(along = sf_fish)){
if(st_intersects(sf_fish[i,],sf_eddy)) sf_fish[i,"vals"] <- sf_eddy$vals}

This is probably an awful way of doing it, obviously.

But is any of this even vaguely a reasonable way of solving this problem, or am I missing much more elegant solutions due to my amateurish knowledge of sf, sp, rgeos, etc? I have 166 fish files with average 286 days of locations = 47476 total lookups. Would it be good/fast to create an index of fish/eddy date match pairs (there may be multiple eddies on the same day) and then only test intersections on those? Since I need to append eddy$vals (actually 2 or 3 columns) to fish, would it be better to use spatial join, perhaps st_join?

Edit: comment & output map oddities:
600,000 total fish/eddy row ggplot output:

Looks like I need to fix the projection as the eddies aren't printing west of 0. They're definitely in the same CRS now; maybe the originals are 0-360 not -180:180. Also while fish boxes print nicely, eddies are too small: mean r=80km=~1deg@45N, and they're definitely printing much smaller than that. I'll have a think about that too. Also: will for() have a problem if 2 eddies exist on the same day?

Edit2: originals were 0:360 which I corrected. Eddy radii still too small:

Per code discussion, original eddy file with latlons was converted with st_as_sf to crs4326 (wgs84) then st_transform(sf_eddy,6931) which is North Atlantic, per image above. geometry column in resultant file is POLYGON, XY dimension, values are in projected coordinates not latlons, so e.g. sf_eddy_buffered$geometry[1] bbox: xmin -4894790 ymin -2418555 xmax -4894648 ymax -2418413. speed_radius is 71 so the difference between xmin & xmax is 142 which is 71*2 which is right. Does radius unit not equate to projection distance unit? Stewart's examples were 81000. Eddy radius unit: km. Projection / geometry unit: https://epsg.io/6931 says units are indeed metre. Multiply radius by 1000 & try again:

So that's all good. Just tried the updated st_join code, think it could use some work - takes absolutely forever probably on account of the 3639209 eddy rows vs 461 fish rows. Also it has created multiple rows for multiple eddies for the same fish on the same day. Potentially a function of the relatively large overlap opportunity given the error buffer box around the fish.
I suspect I'll be able to fix this by marking duplicates and deleting the duplicates which are furthest away. The current way of doing (via v and rbind) it is better than the previous way (direct cell assignment of a value) IMO since otherwise multiple values would be sent to a single cell, either crashing it or silently overwriting.

For speeding up the code I was thinking that I could subset the eddy file to only the overlap in date range of the current fish, and use that subset in the join loop. The join loop already subsets the fish and the eddy files by date anyway so maybe this is redundant, or maybe reducing the size of the eddy data that needs to be worked on will give the CPU/RAM a reduced workload each time?

Edit3: this DRASTICALLY speeds things up. Adding a simple loop counter for thisDate showed it completed at 275 of 461 rows. And produced a 481 length file. Possibly there are lots of days where the fish wasn't in an eddy (thisDate loop results in no v and no rbind), and lots of days of duplicates where the fish was in 2+ eddies (thisDate loop results in multirow v and rbind). Still feels like it should process 461 loops, however...

Edit4: a small tweak:

v <- st_join(thisFishRow, thisEddyRow, left = F, largest = TRUE)

largest means it'll only join the eddy with the largest overlap area, resulting in a max v rowlength of 1 and thus 1 value per day max. However, nuances of the paths mean the fish can seem to go back and forth between different ongoing eddies. My suspicion (based on a similar study) is that they are more likely to stay within a strong anticyclonic eddy system. Radius (and thus area) of eddy controls overlap possibly more than position. For a 1.5 month section I found the highest-overlap eddy track number bounced between 3 eddies. Eddy track number 217008 is the most powerfully anticyclonic. Here's a (crap) plot:

As you can see, 217008 is right in the centre of the fish error box whereas the larger, more diffuse, lower powered eddies 217343 and 39625 are off to the edge. However their larger size often sees them bumped to the top because they have more area to give, and centroid proximity isn't counted.
So: if fishbox overlaps with eddy on same day then include eddy in shortlist
(thisFishRow & thisEddyRow stay the same). Then: Choose eddy from shortlist based on lowest distance from centroids (something instead of st_join). To be continued!

Edit5:

fishNearEddies <- NULL
for (thisDate in sf_df_nona$date) {
 thisFishRow <- sf_df_nona[sf_df_nona$date == thisDate, ]
 thisEddyRow <- sebDateSub[sebDateSub$date == thisDate, ]
 overlapToday <- st_join(thisFishRow, thisEddyRow, left = F)
 if (nrow(overlapToday) > 0) {
  overlapEddies <- thisEddyRow[thisEddyRow$track %in% overlapToday$track,]
  st_distance(x = thisFishRow, y = overlapEddies, by_element = TRUE) #0 0 due to overlap min dist = 0
 } #close if
} #close for

st_distance won't work because the spatial features (buffer box & buffer circle) overlap, so min distance = 0. I ALSO need centroids to test against, I presume?

Edit6: Final Edit with working code, & answer award to Stewart for all his help. Thanks again sir.

# overlap join loop####
fishNearEddies <- NULL
EdCentroidDist <- NULL
counter <- 1
ofhowmany <- length(sf_df_nona$date)
for (thisDate in sf_df_nona$date) {
  thisFishRow <- sf_df_nona[sf_df_nona$date == thisDate, ] #will be 1 row per day
  thisEddyRow <- sebDateSub[sebDateSub$date == thisDate, ] #all eddies that day, could be multi rows
  overlapToday <- st_join(thisFishRow, thisEddyRow, left = F) #will join only if they spatial intersect and (already) time match
  if (nrow(overlapToday) > 0) {
    # now need to join based on closest centroid and NOT on highest overlap
    overlapEddies <- thisEddyRow[thisEddyRow$track %in% overlapToday$track,] #subset TER by overlap tracks
    fishcentroid <- st_centroid(thisFishRow)
    eddycentroid <- st_centroid(overlapEddies)
    fishEdDists <- st_distance(x = fishcentroid, y = eddycentroid, by_element = TRUE) #result vector corresponding to overlapEddies rownumbers
    fishNearEddies <- rbind(fishNearEddies, overlapEddies[which.min(fishEdDists),]) #row index for overlapEddies, no index but has date
    EdCentroidDist <- rbind(EdCentroidDist, as.numeric(min(fishEdDists)))
  } #close if
  print(paste0(counter, " of ", ofhowmany, " fish days"))
  counter <- counter + 1
} #close for

if (!is.null(fishNearEddies)) {  #if there are overlaps, do processing. If not it'll fail

  fishNearEddies %<>% as.data.frame() %>% # convert to nonspatial so I can remove buffer column
    select(-geometry) %>% # remove geometry column. Attributes still remain. Whatever?
    cbind(EdCentroidDist) #add to FNE as column

Body:
Here's something that should get you going:

library(lubridate)
library(sf)
library(ggplot2)

# sample data
fish <- data.frame(
    lat = c(41.1, 43.6, 44.2),
    lon = c(-7, -11, -15),
    date = ymd(c("1990-01-01", "1990-01-02", "1990-01-03"))
)

# Add a blank column for the eddy values we want
fish$vals <- rep(NA, nrow(fish))

eddy <- data.frame(
    lat = c(44, 42.3, 40),
    lon = c(-6, -10.1, -15),
    date = ymd(c("1990-01-01", "1990-01-02", "1990-01-03")),
    track = c(1,1,1),
    radius = c(81000,82000,83000),
    vals = c(11,12,13)
)

# Convert eddy to simple features, using WGS84 CRS
sf_eddy <- st_as_sf(eddy,  coords = c("lon", "lat"), crs=4326)

# Transform from geographical to projected so we can buffer correctly. You might need to pick a different CRS
sf_eddy <- st_transform(sf_eddy, 3035) # ETRS89 / LAEA Europe

# Buffer the eddy points based on their radii
sf_eddy_buffered <- st_buffer(sf_eddy, dist = sf_eddy$radius)

# Add error to fish position. There's probably a better way to do this.
fishErrLat <- 1.9
fishErrLon <- 0.77
fish$buffer <- paste('POLYGON((',
                    fish$lon - fishErrLon, ' ', fish$lat + fishErrLat, ',',
                    fish$lon + fishErrLon, ' ', fish$lat + fishErrLat, ',',
                    fish$lon + fishErrLon, ' ', fish$lat - fishErrLat, ',',
                    fish$lon - fishErrLon, ' ', fish$lat - fishErrLat, ',',

                    fish$lon - fishErrLon, ' ', fish$lat + fishErrLat,
                    '))',
                    sep=''
                )

# Convert fish to simple features, using WGS84 CRS
sf_fish <- st_as_sf(fish, wkt='buffer', crs=4326)

# Transform from geographical to projected
sf_fish <- st_transform(sf_fish, 3035)

#Plot what we've got so far
g <- ggplot() +
        geom_sf(data=sf_eddy_buffered, aes(fill=date)) +
        geom_sf(data=sf_fish, aes(fill=date))

print(g)

# Check for overlap
fishNearEddies <- NULL

for (thisDate in unique(sf_fish$date)) {

    thisFishRow <- sf_fish[sf_fish$date==thisDate, ]
    thisEddyRow <- sf_eddy_buffered[sf_eddy_buffered$date==thisDate, ]

    v <- st_join(thisFishRow, thisEddyRow, left=F)
    if (nrow(v) > 0) {
        fishNearEddies <- rbind(fishNearEddies, v)
    }

}

And check the results:

> fishNearEddies
Simple feature collection with 1 feature and 7 fields
geometry type:  POLYGON
dimension:      XY
bbox:           xmin: 2588691 ymin: 2402830 xmax: 2703342 ymax: 2624501
epsg (SRID):    3035
proj4string:    +proj=laea +lat_0=52 +lon_0=10 +x_0=4321000 +y_0=3210000 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs
   lat lon     date.x     date.y track radius vals                         buffer
2 43.6 -11 1990-01-02 1990-01-02     1  82000   12 POLYGON ((2644792 2624501, ...

This will give you only those fish that intersect with an eddy.


来源: stackexchange

问:
Something akin to "Sparse Fieldsets" in .NET
I'm trying to find the vocabulary to describe what I want and if it exists.

I have a table that shows a few data points from large objects. Loading the entire objects just for the table is very slow. Is there a way to only pass to the front the few properties I want without having to define a new object?

I found something called Sparse Fieldsets in JSON API, and I'm wondering if something like this exists for .NET under another name.

Update:
Talking to another coder, I realize it probably makes more sense to implement something like this between the backend and the database and make a specific call for this table. I still need to work out if I need to create a new object to support this. I think it'd still be faster if I just kept the same object but nulled out all of the connecting objects that I don't need for the table. But maybe that's considered bad practice? Also we're using Entity Framework for what it's worth.

Update 2:
I just created a new query without all of the .Include() and works well enough for what I need:

```<code>_dataContext.ApplePie
.Include(f => f.Apples).ThenInclude(f => f.Apple)
        .Include(f => f.Sugars).ThenInclude(f => f.MolecularStructure)
        .Include(f => f.Recipe)
</code>```

答:
C# is a strongly-typed language and I would say that it is unusual for C# to support scenarios, when object definition (properties) are not known in advance, like in JSON API "fields" parameter case. Implementing this would imply using reflection to filter the properties, which is usually slow and error-prone.

When implementing C# web-services, people usually create one DTO response model per each request.

If your table has fixed set of fields, I would personally recommend to create a DTO class containing only the fields which are required for your table, and then create a method which returns this response for your specific request. While it doesn't align with "without having to define a new object" in the question, it makes the intention clear and makes it really easier to maintain the API in future.

You might want to use libraries like AutoMapper to save time and avoid duplicated code of copying the values from data model to DTO, if you have many such methods.

ContentLicense: CC BY-SA 4.0

Body:
I'm trying to find the vocabulary to describe what I want and if it exists.

I have a table that shows a few data points from large objects. Loading the entire objects just for the table is very slow. Is there a way to only pass to the front the few properties I want without having to define a new object?

I found something called Sparse Fieldsets in JSON API, and I'm wondering if something like this exists for .NET under another name.

Update:
Talking to another coder, I realize it probably makes more sense to implement something like this between the backend and the database and make a specific call for this table. I still need to work out if I need to create a new object to support this. I think it'd still be faster if I just kept the same object but nulled out all of the connecting objects that I don't need for the table. But maybe that's considered bad practice? Also we're using Entity Framework for what it's worth.

Update 2:
I just created a new query without all of the .Include() and works well enough for what I need:

_dataContext.ApplePie
.Include(f => f.Apples).ThenInclude(f => f.Apple)
        .Include(f => f.Sugars).ThenInclude(f => f.MolecularStructure)
        .Include(f => f.Recipe)


问:
Is phase kubeconfig required after phase certs in kubeadm?
I've recently upgraded with kubeadm, which I expect to rotate all certificates, and for good measure, I also ran kubeadm init phase certs all, but I'm not sure what steps are required to verify that the certs are all properly in place and not about to expire.

I've seen a SO answer reference kubeadm init phase kubeconfig all is required in addition, but cannot find in the kubernetes kubeadm documentation telling me that it needs to be used in conjunction with phase certs.

What do I need to do to make sure that the cluster will not encounter expired certificates?

I've tried verifying by connecting to the secure local port: echo -n | openssl s_client -connect localhost:10250 2>&1 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | openssl x509 -text -noout | grep Not, which gives me expirations next month.

While openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text and openssl x509 -in /etc/kubernetes/pki/apiserver-kubelet-client.crt -noout -text yield dates over a year in advance.

These conflicting dates certainly have me concerned that I will find myself like many others with expired certificates. How do I get in front of that?

Thank you for any guidance.

答:
In essence kubeadm init phase certs all regenerates all your certificates including your ca.crt (Certificate Authority), and Kubernetes components use certificate-based authentication to connect to the kube-apiserver (kubelet, kube-scheduler, kube-controller-manager) so you will also have to regenerate pretty much all of those configs by running kubeadm init phase kubeconfig all

Keep in mind that you will have to regenerate the kubelet.conf on all your nodes since it also needs to be updated to connect to the kube-apiserver with the new ca.crt. Also, make sure you add all your hostnames/IP addresses that your kube-apiserver is going to serve on to the kubeadm init phase certs all command (--apiserver-cert-extra-sans)

Most likely you are not seeing the updated certs when connecting through openssl is because you haven't restarted the Kubernetes components and in particular the kube-apiserver.  So you will have to start your kube-apiserver, kube-scheduler, kube-controller-manager, etc (or kube-apiservers, kube-schedulers, etc if you are running a multi-master control plane) You will also have to restart your kubelets on all your nodes.


问:
How to fix TypeError in vue.js created hook?
error:

```<code>vue.js:597 [Vue warn]: Error in created hook: "TypeError: handlers[i].call is not a function"

found in

---> <StageExecs>
</code>```

vue.js

```<code><div id="vue-job">
    <div class="row">
        <h3>test</h3>
        <stage-execs></stage-execs>
    </div>
</div>

<script>
    Vue.component('jobs', {
        delimiters: [ '[[', ']]' ],
        props: ['job'],
        data: function() {
                return {
                    showExecs: false,
                    build_ids: []
                }
            },
        created: {
            stageExecs() {
                url = "api/v2/exec?" + this.job.api_url + "&limit=10"
                fetch(url)
                    .then(response => response.json())
                    .then(body => {
                        for(i=0; i<body.length; i++){
                            this.build_ids.push({
                                'id': JSON.stringify(body[i].build_id),
                            })
                        }
                    })
                    .catch( err => {
                        console.log('Error Fetching:', url, err);
                        return { 'failure': url, 'reason': err };
                    });
                }
        },
        template: `
        <ul id="example-1">
          <li v-for="item in build_ids">
            [[ item.url ]]
          </li>
        </ul>
        `,
    });

var v_root = new Vue({
    delimiters: [ '[[', ']]' ],
    el: '#vue-job',
    data: {
        job_exec: {{ job_exec|safe }},

    }
});
</code>```

I assume something is being called in the incorrect order here, but new to vue.js so not sure what I did wrong.

答:
The problem is created() hook needs to be a method, not an object. See this working sandbox fix https://codesandbox.io/s/vue-template-e0dzj</p>

```<code>created() {
    url = "api/v2/exec?" + this.job.api_url + "&limit=10"
    fetch(url)
        .then(response => response.json())
        .then(body => {
            for(i=0; i<body.length; i++){
                this.build_ids.push({
                    'id': JSON.stringify(body[i].build_id),
                })
            }
        })
        .catch( err => {
            console.log('Error Fetching:', url, err);
            return { 'failure': url, 'reason': err };
        });
    }
},
</code>```

Body:
error:

vue.js:597 [Vue warn]: Error in created hook: "TypeError: handlers[i].call is not a function"

found in

---> <StageExecs>

vue.js

test

        <stage-execs></stage-execs>

Body:
The problem is created() hook needs to be a method, not an object. See this working sandbox fix https://codesandbox.io/s/vue-template-e0dzj

created() {
    url = "api/v2/exec?" + this.job.api_url + "&limit=10"
    fetch(url)
        .then(response => response.json())
        .then(body => {
            for(i=0; i<body.length; i++){
                this.build_ids.push({
                    'id': JSON.stringify(body[i].build_id),
                })
            }
        })
        .catch( err => {
            console.log('Error Fetching:', url, err);
            return { 'failure': url, 'reason': err };
        });
    }
},


问:
dump xml string verbatim to mysql db
I need to dump an xml string (coming into a web service as a POST param), directly into a Mysql DB. The column into which I am writing is currently of type 'text', though I have tried blob as well.

Right now, when I save the string and retrieve it later on using sql, it comes back in a serialized like format, like this:

```<code>a:1:{s:14:"<?xml_encoding";s:1502:"UTF-8?><record>
                <nodes></nodes>
              </record>";}
</code>```

Whereas, I need to parse that xml as a simplexml object, and hence need it to be better formed.

This is what I am doing codewise:

Accept Post Param in web service

Serialize and store it in DB using doctrine ORM, kind of like

$record->xml_rec = serialize($_POST)

Retrieve it and echo it.

Oddly enough, if I unserialize and echo is upon retrial, I get an array. The array looks like this upon print_f

```<code>Array
(
[<?xml_encoding] => UTF-8?><record>
                <nodes></nodes>
              </record>
)
</code>```

Still not parse-able by simplexml.

Is this a mysql storage problem or am I doing something wrong with the POST params? I've tried htmlspecialchars to save the string in the db, but similar problems remain.

any help is appreciated, thanks

答: What you have is a "serialized" php array that you can unserialize by using ... PHP's unserialize function.</p>

ContentLicense: CC BY-SA 2.5

Title: dump xml string verbatim to mysql db

Body:
I need to dump an xml string (coming into a web service as a POST param), directly into a Mysql DB. The column into which I am writing is currently of type 'text', though I have tried blob as well.

Right now, when I save the string and retrieve it later on using sql, it comes back in a serialized like format, like this:

a:1:{s:14:"<?xml_encoding";s:1502:"UTF-8?><record>
                <nodes></nodes>
              </record>";}

Whereas, I need to parse that xml as a simplexml object, and hence need it to be better formed.

This is what I am doing codewise:

Accept Post Param in web service

Serialize and store it in DB using doctrine ORM, kind of like

$record->xml_rec = serialize($_POST)

Retrieve it and echo it.

Oddly enough, if I unserialize and echo is upon retrial, I get an array. The array looks like this upon print_f

Array
(
[<?xml_encoding] => UTF-8?><record>
                <nodes></nodes>
              </record>
)

Still not parse-able by simplexml.

Is this a mysql storage problem or am I doing something wrong with the POST params? I've tried htmlspecialchars to save the string in the db, but similar problems remain.

any help is appreciated, thanks

ContentLicense: CC BY-SA 3.0


问:
What was ALGOL compiler originally programmed in?
I've been look around and I found little info on what was used to create the ALGOL 58 compiler. Was the compiler created similar to FORTRAN in that they used physical cards to create it? </p>

答:
The wording "the Algol 58 compiler" suggests there was only one, but this is not the case; there were quite a few, with varying degrees of fidelity to the published IAL report.

For Don Knuth's compiler for the Burroughs 205, we can look at the listings.


问:
File manager for TinyMCE 5.0.6
I have installed TinyMCE (TinyMCE Dev Package 5.0.6) in my personal project. Everything is installed and
configured correctly, but there is no File Manager available in this package. To resolve this flaw I have tried to install
ResponsiveFileManager (RFM), but it just doesn't work. There is an error in a console when I click an icon of the RFM:

```<code>plugin.js:84 Uncaught TypeError: editor.windowManager.openUrl is not a function
    at Object.openmanager [as onAction] (plugin.js:84)
    at theme.js:21482
    at theme.js:9575
    at theme.js:21480
    at theme.js:2457
    at each (theme.js:267)
    at run (theme.js:2456)
    at theme.js:2969
    at theme.js:56
    at theme.js:10825
</code>```

I guess that RFM is not properly prepared for this version of TinyMCE.

So I want to ask if there are any available file manager plugins for TinyMCE.
I need a file manager which is capable of:
- preview of uploaded photos,
- create new catalogs,
- upload files to server's catalogs (with Drag and Drop functionality as well).

答: Are you sure you're using 5.0.6? editor.windowManager.openUrl was a new API in that release, so if you're using an earlier version it might cause that exception.

Title: File manager for TinyMCE 5.0.6

Body:
I have installed TinyMCE (TinyMCE Dev Package 5.0.6) in my personal project. Everything is installed and
configured correctly, but there is no File Manager available in this package. To resolve this flaw I have tried to install
ResponsiveFileManager (RFM), but it just doesn't work. There is an error in a console when I click an icon of the RFM:

plugin.js:84 Uncaught TypeError: editor.windowManager.openUrl is not a function
    at Object.openmanager [as onAction] (plugin.js:84)
    at theme.js:21482
    at theme.js:9575
    at theme.js:21480
    at theme.js:2457
    at each (theme.js:267)
    at run (theme.js:2456)
    at theme.js:2969
    at theme.js:56
    at theme.js:10825

I guess that RFM is not properly prepared for this version of TinyMCE.

So I want to ask if there are any available file manager plugins for TinyMCE.
I need a file manager which is capable of:
- preview of uploaded photos,
- create new catalogs,
- upload files to server's catalogs (with Drag and Drop functionality as well).


问:
Why are my forecast predictions coming out as NaN?
My problem is pretty simple, and I know I'm missing something very obvious, I just can't figure out what it is....

My test predictions for Holt-Winters are coming out as NaN and I can't figure out why. Can anyone help on this?

I'm using a Jupyter Notebook, and trying to forecast sales of one SKU using Holt-Winters method. I even went as far as using

Here is the code I used:

```<code># Import the libraries needed to execute Holt-Winters

import pandas as pd
import numpy as np
%matplotlib inline

df = pd.read_csv('../Data/M1045_White.csv',index_col='Month',parse_dates=True)

# Set the month column as the index column

df.index.freq = 'MS'
df.index

df.head()

df.info()
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 48 entries, 2015-05-01 to 2019-04-01
Freq: MS
Data columns (total 7 columns):
Sales       48 non-null int64
EWMA12      48 non-null float64
SES12       47 non-null float64
DESadd12    47 non-null float64
DESmul12    47 non-null float64
TESadd12    48 non-null float64
TESmul12    12 non-null float64
dtypes: float64(6), int64(1)
memory usage: 3.0 KB

from statsmodels.tsa.holtwinters import SimpleExpSmoothing

# Train Test Split

train_data = df.iloc[:36] # Goes up to but not including 36
test_data = df.iloc[12:]

# Fit the Model

fitted_model = exponentialSmoothing(train_data['Sales'],trend='mul',seasonal='mul',seasonal_periods=12).fit()

test_predictions = fitted_model.forecast(12).rename('HW M1045 White Forecast')

test_predictions

Here is the output of my predictions:

2018-05-01   NaN
2018-06-01   NaN
2018-07-01   NaN
2018-08-01   NaN
2018-09-01   NaN
2018-10-01   NaN
2018-11-01   NaN
2018-12-01   NaN
2019-01-01   NaN
2019-02-01   NaN
2019-03-01   NaN
2019-04-01   NaN
Freq: MS, Name: HW M1045 White Forecast, dtype: float64
</code>```

Can someone please point out what I may have missed? This seems to be a simple problem with a simple solution, but it's kicking my butt.

Thanks!

答: The answer has something to do with the seasonal_periods variable being set to 12. If this is updated to 6 then the predictions yield actual values. I'm not a stats expert in Exponential Smoothing to understand why this is the case.

Body:
My problem is pretty simple, and I know I'm missing something very obvious, I just can't figure out what it is....

My test predictions for Holt-Winters are coming out as NaN and I can't figure out why. Can anyone help on this?

I'm using a Jupyter Notebook, and trying to forecast sales of one SKU using Holt-Winters method. I even went as far as using

Here is the code I used:

# Import the libraries needed to execute Holt-Winters

import pandas as pd
import numpy as np
%matplotlib inline

df = pd.read_csv('../Data/M1045_White.csv',index_col='Month',parse_dates=True)

# Set the month column as the index column

df.index.freq = 'MS'
df.index

df.head()

df.info()
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 48 entries, 2015-05-01 to 2019-04-01
Freq: MS
Data columns (total 7 columns):
Sales       48 non-null int64
EWMA12      48 non-null float64
SES12       47 non-null float64
DESadd12    47 non-null float64
DESmul12    47 non-null float64
TESadd12    48 non-null float64
TESmul12    12 non-null float64
dtypes: float64(6), int64(1)
memory usage: 3.0 KB

from statsmodels.tsa.holtwinters import SimpleExpSmoothing

# Train Test Split

train_data = df.iloc[:36] # Goes up to but not including 36
test_data = df.iloc[12:]

# Fit the Model

fitted_model = exponentialSmoothing(train_data['Sales'],trend='mul',seasonal='mul',seasonal_periods=12).fit()

test_predictions = fitted_model.forecast(12).rename('HW M1045 White Forecast')

test_predictions

Here is the output of my predictions:

2018-05-01   NaN
2018-06-01   NaN
2018-07-01   NaN
2018-08-01   NaN
2018-09-01   NaN
2018-10-01   NaN
2018-11-01   NaN
2018-12-01   NaN
2019-01-01   NaN
2019-02-01   NaN
2019-03-01   NaN
2019-04-01   NaN
Freq: MS, Name: HW M1045 White Forecast, dtype: float64

Can someone please point out what I may have missed? This seems to be a simple problem with a simple solution, but it's kicking my butt.

Thanks!


问:
Improving data.table subsetting performance
I am running a large monte-carlo simulation, and I discovered that sub-setting/searching my data is the slowest part of my code. In order to test some alternatives, I bench-marked performance with dataframes, data.table, and a matrix.
Here is the benchmark code:

```<code>library(data.table)
#install.packages('profvis')
library(profvis)
x.df = data.frame(a=sample(1:10,10000,replace=T), b=sample(1:10,10000,replace=T)) # set up a dataframe
x.dt = as.data.table(x.df) # a data.table
setkey(x.dt,a) # set key for faster searches
x.mat = as.matrix(x.df) # a matrix

profvis({
for (i in 1:10000) {
  # test simple subsetting
  xsubset.mat = x.mat[100:200,2]
  xsubset.df = x.df[100:200,2]
  xsubset.dt = x.dt[100:200,2]
  # test search preformance
  xsearch.mat = x.mat[which(x.df$a==10),2]
  xsearch.df = x.df[which(x.df$a==10),2]
  xsearch.dt = x.dt[.(10),2]
}
})
</code>```

Here are my results:

In all seriousness, I love the compact syntax of data.table, and I am wondering if there is something I can do to improve its performance. According to the  creators, its supposed to be super fast. Am I using it incorrectly?

答:
After some more benchmarking, I now understand the issue. The fastest package depends on whether I'm doing many small searches or one big search. It seems that data.table has a lot of overhead per search, making it more suited for working with one huge table, not many searches on small ones.

Consider the following code, and compare with the original:

```<code># make a giant table, but search it only once:
x.df = data.frame(a=sample(1:10,100000000,replace=T), b=sample(1:10,100000000,replace=T))
x.dt = as.data.table(x.df)
setkey(x.dt,a)
x.mat = as.matrix(x.df)

profvis({
for (i in 1:1) {
  xsubset.mat = x.mat[100:200,2]
  xsubset.df = x.df[100:200,2]
  xsubset.dt = x.dt[100:200,2]

  xsearch.mat = x.mat[which(x.df$a==10),2]
  xsearch.df = x.df[which(x.df$a==10),2]
  xsearch.dt = x.dt[.(10),2]
}
})
</code>```

Results:

Body:
I am running a large monte-carlo simulation, and I discovered that sub-setting/searching my data is the slowest part of my code. In order to test some alternatives, I bench-marked performance with dataframes, data.table, and a matrix.
Here is the benchmark code:

library(data.table)
#install.packages('profvis')
library(profvis)
x.df = data.frame(a=sample(1:10,10000,replace=T), b=sample(1:10,10000,replace=T)) # set up a dataframe
x.dt = as.data.table(x.df) # a data.table
setkey(x.dt,a) # set key for faster searches
x.mat = as.matrix(x.df) # a matrix

profvis({
for (i in 1:10000) {
  # test simple subsetting
  xsubset.mat = x.mat[100:200,2]
  xsubset.df = x.df[100:200,2]
  xsubset.dt = x.dt[100:200,2]
  # test search preformance
  xsearch.mat = x.mat[which(x.df$a==10),2]
  xsearch.df = x.df[which(x.df$a==10),2]
  xsearch.dt = x.dt[.(10),2]
}
})

Here are my results:

In all seriousness, I love the compact syntax of data.table, and I am wondering if there is something I can do to improve its performance. According to the  creators, its supposed to be super fast. Am I using it incorrectly?

Body:
After some more benchmarking, I now understand the issue. The fastest package depends on whether I'm doing many small searches or one big search. It seems that data.table has a lot of overhead per search, making it more suited for working with one huge table, not many searches on small ones.

Consider the following code, and compare with the original:

# make a giant table, but search it only once:
x.df = data.frame(a=sample(1:10,100000000,replace=T), b=sample(1:10,100000000,replace=T))
x.dt = as.data.table(x.df)
setkey(x.dt,a)
x.mat = as.matrix(x.df)

profvis({
for (i in 1:1) {
  xsubset.mat = x.mat[100:200,2]
  xsubset.df = x.df[100:200,2]
  xsubset.dt = x.dt[100:200,2]

  xsearch.mat = x.mat[which(x.df$a==10),2]
  xsearch.df = x.df[which(x.df$a==10),2]
  xsearch.dt = x.dt[.(10),2]
}
})

Results:


问:
to replace using find word in array from text file
I have problem that str_replace can't replace the second to last word... first word is working then second up to last is not work in array.. I tried enter the message "ducker apple banana chicken" in "chat.php?message=" then become like this:

output:

"***er apple banana chicken"

badwords.txt:

```<code>duck;
apple;
banana;
chicken;
water;
</code>```

chat.php

```<code>$censoredfilter = file("badwords.txt");
$censoredfilter = implode("\n", $censoredfilter);
$censoredfilter = explode(";", $censoredfilter);

$message = $_GET['message'];

for($i = 0; $i<sizeof($censoredfilter);$i++)
{
    $message = str_replace($censoredfilter[$i], "***", $message);
}
</code>```

答:
To add to @dogmatic69's answer I'd also consider removing the ;'s from your badwords.txt file and shorten the code to just this:

```<code>$censoredfilter = file_get_contents("badwords.txt");
$censoredfilter = explode("\n", $censoredfilter);

$message = str_replace($censoredfilter, '***', $_GET['message']);
</code>```

Also be aware that if badwords.txt is editable by anyone else their text editor may use different line endings e.g. \r\n, or \r causing your script to fail.

Body:
I have problem that str_replace can't replace the second to last word... first word is working then second up to last is not work in array.. I tried enter the message "ducker apple banana chicken" in "chat.php?message=" then become like this:

output:

"***er apple banana chicken"

badwords.txt:

duck;
apple;
banana;
chicken;
water;

chat.php

$censoredfilter = file("badwords.txt");
$censoredfilter = implode("\n", $censoredfilter);
$censoredfilter = explode(";", $censoredfilter);

$message = $_GET['message'];

for($i = 0; $i<sizeof($censoredfilter);$i++)
{
    $message = str_replace($censoredfilter[$i], "***", $message);
}

Body:
To add to @dogmatic69's answer I'd also consider removing the ;'s from your badwords.txt file and shorten the code to just this:

$censoredfilter = file_get_contents("badwords.txt");
$censoredfilter = explode("\n", $censoredfilter);

$message = str_replace($censoredfilter, '***', $_GET['message']);

Also be aware that if badwords.txt is editable by anyone else their text editor may use different line endings e.g. \r\n, or \r causing your script to fail.


问:
Count across dataframe columns based on str.contains (or similar)
I would like to count the number of cells within each row that contain a particular character string, cells which have the particular string more than once should be counted once only.

I can count the number of cells across a row which equal a given value, but when I expand this logic to use str.contains, I have issues, as shown below

d = {'col1': ["a#", "b","c#"], 'col2': ["a", "b","c#"]}
df = pd.DataFrame(d)

#can correctly count across rows using equality
thisworks =( df =="a#" ).sum(axis=1)

#can count across  a column using str.contains
thisworks1=df['col1'].str.contains('#').sum()

#but cannot use str.contains with a dataframe so what is the alternative
thisdoesnt =( df.str.contains('#') ).sum(axis=1)
```

Output should be a series showing the number of cells in each row that contain the given character string.

答:
Something like this should work:

df = pd.DataFrame({'col1': ['#', '0'], 'col2': ['#', '#']})
df['totals'] = df['col1'].str.contains('#', regex=False).astype(int) +\
               df['col2'].str.contains('#', regex=False).astype(int)
df
#   col1 col2  totals
# 0    #    #       2
# 1    0    #       1
```

It should generalize to as many columns as you want.

Body:
I would like to count the number of cells within each row that contain a particular character string, cells which have the particular string more than once should be counted once only.

I can count the number of cells across a row which equal a given value, but when I expand this logic to use str.contains, I have issues, as shown below

d = {'col1': ["a#", "b","c#"], 'col2': ["a", "b","c#"]}
df = pd.DataFrame(d)

#can correctly count across rows using equality
thisworks =( df =="a#" ).sum(axis=1)

#can count across  a column using str.contains
thisworks1=df['col1'].str.contains('#').sum()

#but cannot use str.contains with a dataframe so what is the alternative
thisdoesnt =( df.str.contains('#') ).sum(axis=1)

Output should be a series showing the number of cells in each row that contain the given character string.

Body:
Something like this should work:

df = pd.DataFrame({'col1': ['#', '0'], 'col2': ['#', '#']})
df['totals'] = df['col1'].str.contains('#', regex=False).astype(int) +\
               df['col2'].str.contains('#', regex=False).astype(int)
df
#   col1 col2  totals
# 0    #    #       2
# 1    0    #       1

It should generalize to as many columns as you want.


问:
Calculations for ordered observations using mutate() based on conditional statements
I am trying to calculate growth rates between the specific rows after grouping by another variable (similar to other users).

Here is a sample of my data:

```<code>squirrel_id    wt   age    trialdate
   22639      9.7     0    2017-04-20
   22639      45.9   24    2017-05-14
   22639     130     53    2017-06-12 #caught 3x, 1 trial
   22640     10.3    0     2017-04-20
   22640     49.2    24    2017-05-14
   22640     121     52    2017-06-11
   22640     196     84    2017-07-13 #caught 4x, 2 trials
   23943     12.9     1    2018-04-27
   23943     57.2    26    2018-05-23 #caught 2x, 1 trial
   23760     150     73    2018-06-18
   23760     165     84    2018-06-29 #caught 2x, 2 trials
</code>```

To get this data in this form, I made sure to arrange_by(squirrel_id) first using the library(dplyr) package.

What I am trying to do is calculate the growth rate between:

last observation and second last observation divided by time that has passed (last_wt-second_last_wt/last_age-second_last_age): [using squirrel_id 22640: (196-121)/(84-52)] and then add a column called “trial” that says “2”

second last observation and third last observation divided by time that has passed (second_last_wt-third_last_wt/second_last_age-third_last_age): [using squirrel_id 22640: (121-49.2)/(52-24)] and then add a column called “trial” that says “1”

There is a catch though:

if a squirrel_id is seen 3 times or less in total (like squirrel_id 22639 and 23943), then they would usually only have had 1 trial and therefore 1 growth rate calculation.

BUT, if 2 of the observations are with an age > 40 days old (like squirrel_id 23760), then they had 2 trials.

My hope is to have a final dataset that looks like:

```<code>squirrel_id    wt   age    trialdate    g.rate     trial
   22639      9.7     0    2017-04-20   NA         NA
   22639      45.9   24    2017-05-14   NA         NA
   22639     130     53    2017-06-12   3.0        1     #caught 3x, 1 trial
   22640     10.3    0     2017-04-20   NA         NA
   22640     49.2    24    2017-05-14   NA         NA
   22640     121     52    2017-06-11   2.6         1
   22640     196     84    2017-07-13   2.3         2     #caught 4x, 2 trials
   23943     12.9     1    2018-04-27   NA         NA
   23943     57.2    26    2018-05-23   1.7         1     #caught 2x, 1 trial
   23760     150     73    2018-06-18    NA        1
   23760     165     84    2018-06-29    1.4        2     #caught 2x, 2 trials
</code>```

I’d prefer a dplyr() solution, if possible.

Body:
I am trying to calculate growth rates between the specific rows after grouping by another variable (similar to other users).

Here is a sample of my data:

squirrel_id    wt   age    trialdate
   22639      9.7     0    2017-04-20
   22639      45.9   24    2017-05-14
   22639     130     53    2017-06-12 #caught 3x, 1 trial
   22640     10.3    0     2017-04-20
   22640     49.2    24    2017-05-14
   22640     121     52    2017-06-11
   22640     196     84    2017-07-13 #caught 4x, 2 trials
   23943     12.9     1    2018-04-27
   23943     57.2    26    2018-05-23 #caught 2x, 1 trial
   23760     150     73    2018-06-18
   23760     165     84    2018-06-29 #caught 2x, 2 trials

To get this data in this form, I made sure to arrange_by(squirrel_id) first using the library(dplyr) package.

What I am trying to do is calculate the growth rate between:

last observation and second last observation divided by time that has passed (last_wt-second_last_wt/last_age-second_last_age): [using squirrel_id 22640: (196-121)/(84-52)] and then add a column called “trial” that says “2”

second last observation and third last observation divided by time that has passed (second_last_wt-third_last_wt/second_last_age-third_last_age): [using squirrel_id 22640: (121-49.2)/(52-24)] and then add a column called “trial” that says “1”

There is a catch though:

if a squirrel_id is seen 3 times or less in total (like squirrel_id 22639 and 23943), then they would usually only have had 1 trial and therefore 1 growth rate calculation.

BUT, if 2 of the observations are with an age > 40 days old (like squirrel_id 23760), then they had 2 trials.

My hope is to have a final dataset that looks like:

squirrel_id    wt   age    trialdate    g.rate     trial
   22639      9.7     0    2017-04-20   NA         NA
   22639      45.9   24    2017-05-14   NA         NA
   22639     130     53    2017-06-12   3.0        1     #caught 3x, 1 trial
   22640     10.3    0     2017-04-20   NA         NA
   22640     49.2    24    2017-05-14   NA         NA
   22640     121     52    2017-06-11   2.6         1
   22640     196     84    2017-07-13   2.3         2     #caught 4x, 2 trials
   23943     12.9     1    2018-04-27   NA         NA
   23943     57.2    26    2018-05-23   1.7         1     #caught 2x, 1 trial
   23760     150     73    2018-06-18    NA        1
   23760     165     84    2018-06-29    1.4        2     #caught 2x, 2 trials

I’d prefer a dplyr() solution, if possible.

Body:
Here is another approach using tidyverse:

library(tidyverse)

df %>%
  arrange(squirrel_id, age) %>%
  group_by(squirrel_id) %>%
  mutate(trial = case_when(all(tail(age, 2) > 40) ~ c(rep(NA, n() - 2), 1, 2),
                           TRUE ~ c(rep(NA, n() - 1), 1)),
         g.rate = replace((wt -lag(wt)) / (age - lag(age)),
                           1:(match(1, trial) - 1), NA)) %>%
  ungroup()
#> # A tibble: 11 x 6
#>    squirrel_id    wt   age trialdate  trial g.rate
#>          <dbl> <dbl> <dbl> <chr>      <dbl>  <dbl>
#>  1       22639   9.7     0 2017-04-20    NA  NA
#>  2       22639  45.9    24 2017-05-14    NA  NA
#>  3       22639 130      53 2017-06-12     1   2.9
#>  4       22640  10.3     0 2017-04-20    NA  NA
#>  5       22640  49.2    24 2017-05-14    NA  NA
#>  6       22640 121      52 2017-06-11     1   2.56
#>  7       22640 196      84 2017-07-13     2   2.34
#>  8       23760 150      73 2018-06-18     1  NA
#>  9       23760 165      84 2018-06-29     2   1.36
#> 10       23943  12.9     1 2018-04-27    NA  NA
#> 11       23943  57.2    26 2018-05-23     1   1.77


问:
Firestore Rules break if the document path has a comma
So, I am building a multi tenant application which involves multitude of collections and user invitation. Since I don't know which user ID the signup is going to use and I want the companies to start making changes and updates to their user data I am pre setting the user ID and I use it in several paths as:

```<code>/companies/{company}/customers/{userId}
/users/{userId}
</code>```

Now, this userId is pretty much the email with the dot replaced by a comma. The dot is a character not allowed in firebase but the comma is, and viceversa with emails so it makes sense and there is no problem there.

The issue is when I need to make some rules. For example I have this in the configuration since I want admins to access every company:

```<code>  function cleanEmail(){
        return request.auth.token.email.split('.').join(',') //Here I also tried %2C
    }

    function isSuperadmin() {
      return exists(/databases/$(database)/documents/admins/$(cleanEmail()))
    }

 match /companies/{company} {
      allow write: if isSuperadmin();
      allow read: if isSuperadmin() || belongsToCompany(company)
}
</code>```

The simulator was breaking but I could not understand why, so I even sent a bug report. Eventually I tried making the path hard coded and then I found the problem:

```<code>function isSuperadmin() {
      return exists(/databases/$(database)/documents/admins/test,account@gmail,com) //Same issue with get()
    }
</code>```

I tried with exists() because I thought it could be a bug with get() but the issue remains. I believe this should be a bug since that is a valid Firestore path and I have seen some people already using this "clean email" strategy.

Funny thing is that this was not happening during first testing and I realized that this bug only happens if the comma is BEFORE the @. If you remove the one before that, and leave the second one, seems to work:

I might add a new step to the clean email that turns that into base64 and that might work tho. If someone has a solution great.

答:
I do something similar that looks like this in outline:

```<code>/invitation
    /fb-generated-doc-name
        email: somecustomer@gmail.com
        trip: db-ref-to-trip

/usertrips
    /doc-name-is-UID
        trip: db-ref-to-trip

/trips
   /fb-generated-doc-name
       { ... describes the trip ... }
</code>```

The outline for onCreate auth'd user is:

```<code>exports.authDidCreateUser = functions.auth.user().onCreate((authUser, context) => {
  /*
      find invitation where email == authUser.email
      let docref = collection('usertrips').doc(authUser.uid)
      docref.set({ trip: invitation.trip })
  */
});
</code>```

Body:
So, I am building a multi tenant application which involves multitude of collections and user invitation. Since I don't know which user ID the signup is going to use and I want the companies to start making changes and updates to their user data I am pre setting the user ID and I use it in several paths as:

/companies/{company}/customers/{userId}
/users/{userId}

Now, this userId is pretty much the email with the dot replaced by a comma. The dot is a character not allowed in firebase but the comma is, and viceversa with emails so it makes sense and there is no problem there.

The issue is when I need to make some rules. For example I have this in the configuration since I want admins to access every company:

  function cleanEmail(){
        return request.auth.token.email.split('.').join(',') //Here I also tried %2C
    }

    function isSuperadmin() {
      return exists(/databases/$(database)/documents/admins/$(cleanEmail()))
    }

 match /companies/{company} {
      allow write: if isSuperadmin();
      allow read: if isSuperadmin() || belongsToCompany(company)
}

The simulator was breaking but I could not understand why, so I even sent a bug report. Eventually I tried making the path hard coded and then I found the problem:

function isSuperadmin() {
      return exists(/databases/$(database)/documents/admins/test,account@gmail,com) //Same issue with get()
    }

I tried with exists() because I thought it could be a bug with get() but the issue remains. I believe this should be a bug since that is a valid Firestore path and I have seen some people already using this "clean email" strategy.

Funny thing is that this was not happening during first testing and I realized that this bug only happens if the comma is BEFORE the @. If you remove the one before that, and leave the second one, seems to work:

I might add a new step to the clean email that turns that into base64 and that might work tho. If someone has a solution great.

Body:
I do something similar that looks like this in outline:

/invitation
    /fb-generated-doc-name
        email: somecustomer@gmail.com
        trip: db-ref-to-trip

/usertrips
    /doc-name-is-UID
        trip: db-ref-to-trip

/trips
   /fb-generated-doc-name
       { ... describes the trip ... }

The outline for onCreate auth'd user is:

exports.authDidCreateUser = functions.auth.user().onCreate((authUser, context) => {
  /*
      find invitation where email == authUser.email
      let docref = collection('usertrips').doc(authUser.uid)
      docref.set({ trip: invitation.trip })
  */
});


问:
Install/test problem with full Nativescript CLI install
I've done an apparently clean install of the Nativescript CLI on MacOS X, yet when I run tns doctor it fails: </p>

```<code>bash-3.2$ /usr/local/lib/node_modules/nativescript/bin/tns doctor

/usr/local/lib/node_modules/nativescript/lib/common/verify-node-    version.js:32
            console.warn((`${os.EOL}${nodeWarning.message}${os.EOL}`).yellow.bold)
                  ^
SyntaxError: Unexpected token ILLEGAL
    at Module._compile (module.js:439:25)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object.<anonymous>     (/usr/local/lib/node_modules/nativescript/bin/tns:8:1)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
</code>```

答:
The former problem was due to an obsolete version of node,js; installation of node 10 (LTS) solved it.

The latter problem was due to a writhing nest of pythons - the command 'python' linked to python 2.5, while the install script invoked python 2.7 and its attendant user addons. Relinking 'python' to the python 2.7 executable fixed the issue.

Body:
I've done an apparently clean install of the Nativescript CLI on MacOS X, yet when I run tns doctor it fails:

bash-3.2$ /usr/local/lib/node_modules/nativescript/bin/tns doctor

/usr/local/lib/node_modules/nativescript/lib/common/verify-node-    version.js:32
            console.warn((`${os.EOL}${nodeWarning.message}${os.EOL}`).yellow.bold)
                  ^
SyntaxError: Unexpected token ILLEGAL
    at Module._compile (module.js:439:25)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object.<anonymous>     (/usr/local/lib/node_modules/nativescript/bin/tns:8:1)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)


问:
Shortcut for changing font size
Does anybody know if there is a shortcut or something similar for changing the font size in the text editor in Visual Studio 2008?

What I mean is something like Ctrl + mouse wheel like in Firefox.

Accessing the Options window and then selecting Environment > Fonts and Colors is fine if you don't have to change the font size all the time, but in my case, I would like to be able to change it depending on what I am doing (writing and reading code or just reading it).

I guess I could create two almost identical settings files and import them whenever I want, but that is even slower.

答:
This worked for me:

Ctrl + -     to minimize

Ctrl + +     to maximize

Title: Shortcut for changing font size


问:
What is a working method for extracting numeric values with associated data from open text?
I tried to look for a solution but nothing was giving me quite what I needed. I'm not sure regex can do what I need.

I need to process a large amount of data where license information is provided. I just need to grab the number of licenses and the name for each license then group and tally the license counts for each company.

Here's an example of the data pulled:

```<code>L00129A578-E105C1D138   1 Centralized Recording

$42.00
L00129A677-213DC6D60E   1 Centralized Recording

$42.00
1005272AE2-C1D6CACEC8   5 Station

$45.00
100525B658-3AC4D2C93A   5 Station

$45.00
</code>```

I would need to grab the license count and license name then add like objects so it would grab (1 Centralized Recording, 1 Centralized Recording, 5 Station, 5 Station) then add license counts and output (2 Centralized Recording, 10 Station)

What would be the easiest way to implement this?

答:
```<code>import re, io, pandas as pd
a = open('your_data_file.txt')
pd.read_csv(io.StringIO(re.sub(r'(?m).*\s(\d+)\s+(.*\S+)\s+$\n|.*','\\1,\\2',a)),
                header=None).groupby(1).sum()[0].to_dict()
</code>```

Body:
I tried to look for a solution but nothing was giving me quite what I needed. I'm not sure regex can do what I need.

I need to process a large amount of data where license information is provided. I just need to grab the number of licenses and the name for each license then group and tally the license counts for each company.

Here's an example of the data pulled:

L00129A578-E105C1D138   1 Centralized Recording

$42.00
L00129A677-213DC6D60E   1 Centralized Recording

$42.00
1005272AE2-C1D6CACEC8   5 Station

$45.00
100525B658-3AC4D2C93A   5 Station

$45.00

I would need to grab the license count and license name then add like objects so it would grab (1 Centralized Recording, 1 Centralized Recording, 5 Station, 5 Station) then add license counts and output (2 Centralized Recording, 10 Station)

What would be the easiest way to implement this?


问:
C#/Azure function: Fastest way to serve a file?
I have an Azure Function that serves a file with a custom name:

I'm doing this:

```<code>string outputName = fileName.Substring(0,fileName.Length-4)+"."+clientKey+".msi";

    try
    {
        var res = new HttpResponseMessage(HttpStatusCode.OK);
        var fs = File.Open(filePath, FileMode.Open);
        res.Content = new StreamContent(fs);
        res.Content.Headers.ContentType = new MediaTypeHeaderValue("application/x-msi");
        res.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
        {
            FileName = outputName
        };
        return res;
    }
    catch
    {
        return new HttpResponseMessage(HttpStatusCode.NotFound);
    }
</code>```

Is this the fastest way?

答: That is going to be the best way to serve a file. If the files are extremely large, you may need to add res.Headers.TransferEncodingChunked = true; to help keep the download from failing.

Body:
I have an Azure Function that serves a file with a custom name:

I'm doing this:

string outputName = fileName.Substring(0,fileName.Length-4)+"."+clientKey+".msi";

    try
    {
        var res = new HttpResponseMessage(HttpStatusCode.OK);
        var fs = File.Open(filePath, FileMode.Open);
        res.Content = new StreamContent(fs);
        res.Content.Headers.ContentType = new MediaTypeHeaderValue("application/x-msi");
        res.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
        {
            FileName = outputName
        };
        return res;
    }
    catch
    {
        return new HttpResponseMessage(HttpStatusCode.NotFound);
    }

Is this the fastest way?


问:
How send data from a component to a Module?
I have a component where I have flagPerfil variable, I need to send this variable to my module.

my service

```<code>export class ListUserService {
    public flagPerfil = boolean;
    changeFlagPerfil() {
        console.log('EEE' + this.flagPerfil);
        this.flagPerfil ? this.flagPerfil = false : this.flagPerfil = true;
    }
</code>```

component class:

```<code>@Component({
    selector: 'app-list-user',
    templateUrl: './list-user.component.html',
    styleUrls: ['./list-user.component.scss'],
    providers: [ListUserService]
})
export class ListUserComponent implements OnInit {
    private flagPerfil = false;

    getFlag() {
        return this.flagPerfil;
    }
</code>```

This is the view list-user.component.html

```<code><span *ngIf="!flagPerfil">
  <input type="button" id="usuarios" name="more" (click)="changeFlagPerfil()"
    value="Amigos">
</span>
</code>```

I have a module, I want to inject the service in the module, because
I need the FlagProfile variable

```<code>  @NgModule({
    imports: [
        IonicModule,
        CommonModule,
        FormsModule,
        RouterModule.forChild([{path: '', component: Tab1Page}])
    ],
    declarations: [Tab1Page, ListUserComponent, FriendsComponent]
})
export class Tab1PageModule {
    public flagPerfil: boolean;

    constructor(private listUserService: ListUserService) {
        this.flagPerfil = listUserService.flagPerfil;
        console.log('FLAG', listUserService.flagPerfil);
    }
}
</code>```

How do I use an event Emit?

I need the module to automatically hear the changes of that variable

答:
```<code>export class ListUserService {
    public flag$ = new Subject< boolean >();

}

export class Tab1PageModule {
    public flagPerfil: boolean;

    constructor(private listUserService: ListUserService) {
        listUserService.$flags.subscribe((flag)=>{
           console.log('FLAG', flag;
           this.flagPerfil = flag;
      });

    }
}

export class ListUserComponent implements OnInit {
    private flagPerfil = false;

    getFlag() {
        return this.flagPerfil;
    }

    constructor(private listUserService: ListUserService) {

    }
    functionThatGetsCalledWhenTheFlagIsChanged(){
       this.listUserService.flags$.next(this.flagPerfil);

     }

}

</code>```

Body:
I have a component where I have flagPerfil variable, I need to send this variable to my module.

my service

export class ListUserService {
    public flagPerfil = boolean;
    changeFlagPerfil() {
        console.log('EEE' + this.flagPerfil);
        this.flagPerfil ? this.flagPerfil = false : this.flagPerfil = true;
    }

component class:

@Component({
    selector: 'app-list-user',
    templateUrl: './list-user.component.html',
    styleUrls: ['./list-user.component.scss'],
    providers: [ListUserService]
})
export class ListUserComponent implements OnInit {
    private flagPerfil = false;

    getFlag() {
        return this.flagPerfil;
    }

This is the view list-user.component.html

  <input type="button" id="usuarios" name="more" (click)="changeFlagPerfil()"
    value="Amigos">

I have a module, I want to inject the service in the module, because
I need the FlagProfile variable

  @NgModule({
    imports: [
        IonicModule,
        CommonModule,
        FormsModule,
        RouterModule.forChild([{path: '', component: Tab1Page}])
    ],
    declarations: [Tab1Page, ListUserComponent, FriendsComponent]
})
export class Tab1PageModule {
    public flagPerfil: boolean;

    constructor(private listUserService: ListUserService) {
        this.flagPerfil = listUserService.flagPerfil;
        console.log('FLAG', listUserService.flagPerfil);
    }
}

How do I use an event Emit?

I need the module to automatically hear the changes of that variable


问:
How to Dynamically Autofill Cells
I would like to dynamically fill a column. However, with the VBA code I have now, the formula fills all the way to the max row count (1.04 million). I would like it to stop at the max rows found in the column to its left (same as manual auto fill)

```<code>Sub Vlookup()

Range("C2").Select
ActiveCell.FormulaR1C1 =
"=IFERROR(VLOOKUP(RC[-2],'C:\Test.xlsx'!R2C6:R1000C15,10,FALSE),"""")"
Selection.AutoFill Destination:=Range("C2", Range("C2").End(xlDown))

End Sub
</code>```

I tried to use

```<code>Selection.Autofill Destination :=
Range("C2",Selection.Range("C2").End(xldown))
</code>```

But that didnt seem to work either

答:
first you can get the last row coming from column B and use that as reference for auto fill. see my example below.</p>

```<code>Dim lastRow As Integer
Range("B1").Select

Selection.End(xlDown).Select
lastRow = Selection.Row
Range("C2").Select
Selection.AutoFill Destination:=Range("C2:C" & lastRow)
</code>```

Title: How to Dynamically Autofill Cells

Body:
I would like to dynamically fill a column. However, with the VBA code I have now, the formula fills all the way to the max row count (1.04 million). I would like it to stop at the max rows found in the column to its left (same as manual auto fill)

Sub Vlookup()

Range("C2").Select
ActiveCell.FormulaR1C1 =
"=IFERROR(VLOOKUP(RC[-2],'C:\Test.xlsx'!R2C6:R1000C15,10,FALSE),"""")"
Selection.AutoFill Destination:=Range("C2", Range("C2").End(xlDown))

End Sub

I tried to use

Selection.Autofill Destination :=
Range("C2",Selection.Range("C2").End(xldown))

But that didnt seem to work either

Body:
first you can get the last row coming from column B and use that as reference for auto fill. see my example below.

Dim lastRow As Integer
Range("B1").Select

Selection.End(xlDown).Select
lastRow = Selection.Row
Range("C2").Select
Selection.AutoFill Destination:=Range("C2:C" & lastRow)


问:
Test jsonb array to see if it contains one or more elements?
I have a user table with a data jsonb column. In that, I have a cars array that can have 0-N cars in it.

I'm trying to do a query like this to find if the user's cars array contains either Toyota or Mitsubishi, but it's throwing an error

```<code>await knex('users.record')
    .where('created_at', '>', '2019-05-21')
    .select('data')
    .whereRaw('cars', ['Toyota', 'Mitsubishi']) // this seems wrong
    .returning('*');
</code>```

One example user's cars array is ['Toyota', 'Mitsubishi', 'Honda', 'Ford']

答:
Try replacing

```<code>.whereRaw('cars', ['Toyota', 'Mitsubishi'])
</code>```

with

```<code>where('cars', 'like','%Toyota%').orWhere('cars', ,like, '%Mitsubishi%')
</code>```

Body:
I have a user table with a data jsonb column. In that, I have a cars array that can have 0-N cars in it.

I'm trying to do a query like this to find if the user's cars array contains either Toyota or Mitsubishi, but it's throwing an error

await knex('users.record')
    .where('created_at', '>', '2019-05-21')
    .select('data')
    .whereRaw('cars', ['Toyota', 'Mitsubishi']) // this seems wrong
    .returning('*');

One example user's cars array is ['Toyota', 'Mitsubishi', 'Honda', 'Ford']

Body:
Try replacing

.whereRaw('cars', ['Toyota', 'Mitsubishi'])

with

where('cars', 'like','%Toyota%').orWhere('cars', ,like, '%Mitsubishi%')


问:
Enter a docker container running with Google Cloud Run
Is it possible to enter a container powered by Google Cloud Run? Something in the manner of docker exec -it CONTAINER /bin/bash?

I'm facing a bug i can't reproduce running a container based on the very same image neither locally nor using Google Cloud Shell to run that container.

I'm not picky regarding whether it comes to using Cloud Shell to connect to Cloud Run or doing so from my local environment.

答:
Is it possible to enter a container powered by Google Cloud Run?

No, the only port you can access the service is $PORT (currently 8080) and only via HTTP/HTTPS requests.

You will need to debug the old fashioned way: logging statements to Google Stackdriver - which is already built-in to Cloud Run, just do "logging.info(...)" for your language.

If your problem is container startup, check Stackdriver for error messages.


问:
Jquery ajax call to code behind method in user control
Can we call User control code behind method using Jquery ajax ?

Thanks,

答:
"Can we call User control code behind method using Jquery ajax ?

If you mean using Web Methods, no. You can't directly access a method of a user control through an external query the way you can with a PageMethod. They are only supported at the the page (ascx) level.

Just use a WebService asmx for your functionality instead.


问:
Overwrite file if new session but append if same session
I have a function to convert any write-hosts in the script (keeping them for coloring purposes later) to tee object

```<code>function Write-Host($object)
{
    $object | tee .\LOG.txt -Append
}
</code>```

This currently appends to the file if it exists (or creates one if it doesnt)

i want to create one everytime a NEW session is started. only append to the file if you are in the same session

how do i do that?

答:
I'll offer an alternative method for future searchers. You can get what time your current process started at, compare it to when the file was created, and delete the file if it was created before your current PID was started.

```<code>function Write-Host
{
[cmdletbinding()]
Param([parameter(ValueFromPipeline=$true)]$object)
Begin{
    $allObjects=New-Object System.Collections.ArrayList
}
Process{
    $allObjects.Add($object)|out-null
}
End{
    if((test-path .\log.txt) -and (get-item '.\LOG.txt').CreationTime -lt (Get-Process -Id $PID).StartTime){Remove-Item .\LOG.txt}
    $allObjects | tee .\LOG.txt -Append
}
}
</code>```

Plus with the conversion to an advanced function you can just pipe things to it, such as:

```<code>Get-Process | Write-Host
</code>```

Or I suppose the simplest thing to do is just delete the log file as the first thing your script does.

Body:
I have a function to convert any write-hosts in the script (keeping them for coloring purposes later) to tee object

function Write-Host($object)
{
    $object | tee .\LOG.txt -Append
}

This currently appends to the file if it exists (or creates one if it doesnt)

i want to create one everytime a NEW session is started. only append to the file if you are in the same session

how do i do that?

Body:
I'll offer an alternative method for future searchers. You can get what time your current process started at, compare it to when the file was created, and delete the file if it was created before your current PID was started.

function Write-Host
{
[cmdletbinding()]
Param([parameter(ValueFromPipeline=$true)]$object)
Begin{
    $allObjects=New-Object System.Collections.ArrayList
}
Process{
    $allObjects.Add($object)|out-null
}
End{
    if((test-path .\log.txt) -and (get-item '.\LOG.txt').CreationTime -lt (Get-Process -Id $PID).StartTime){Remove-Item .\LOG.txt}
    $allObjects | tee .\LOG.txt -Append
}
}

Plus with the conversion to an advanced function you can just pipe things to it, such as:

Get-Process | Write-Host

Or I suppose the simplest thing to do is just delete the log file as the first thing your script does.


问:
Variable is empty inside of prepareforsegue function
I have data that I want to send to another view controller via a segue. I add some user selected data to an array in my first view controller inside of 'didselectrow' for a tableview, and in 'prepareforsegue' I attempt to set the first element of that array equal to a variable in my second view controller. However, I get an 'index out of range error' telling me that the array is empty, and I am not sure why.

Here is the first viewcontroller:

var selectedStores = [String]()

func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) {
        selectedStores.append(stores[indexPath.row])
        performSegue(withIdentifier: "showStoreDetailsSegue", sender: self)
    }

override func prepare(for segue: UIStoryboardSegue, sender: Any?) {
        print("Prepare for segue called")
        if segue.identifier == "showStoreDetailsSegue" {
            let destinationVC = segue.destination as! StoreDetailsViewController

            destinationVC.selectedStore = selectedStores[0]
        }
    }
```

<p>Second viewcontroller (StoreDetailsViewController):</p>

<pre class="lang-swift prettyprint-override"><code>var selectedStore = String()
</code>```

I have another viewcontroller doing the exact same thing, but the array is not empty inside the prepareforsegue function. I have the tableview cell connected to the secondviewcontroller through a segue called "showStoreDetailsSegue".

答:
I have the tableview cell connected to the secondviewcontroller through a segue called "showStoreDetailsSegue".

That’s the problem. You are performing the segue twice, and the first time is before didSelect is called.

One solution: Connect the segue from the view controller, not the cell.

Alternatively (and better), delete your didSelect implementation itself. You don’t really need selectedStores. Just use prepare to find out what the selection is, and go from there.

Body:
I have data that I want to send to another view controller via a segue. I add some user selected data to an array in my first view controller inside of 'didselectrow' for a tableview, and in 'prepareforsegue' I attempt to set the first element of that array equal to a variable in my second view controller. However, I get an 'index out of range error' telling me that the array is empty, and I am not sure why.

Here is the first viewcontroller:

var selectedStores = [String]()

func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) {
        selectedStores.append(stores[indexPath.row])
        performSegue(withIdentifier: "showStoreDetailsSegue", sender: self)
    }

override func prepare(for segue: UIStoryboardSegue, sender: Any?) {
        print("Prepare for segue called")
        if segue.identifier == "showStoreDetailsSegue" {
            let destinationVC = segue.destination as! StoreDetailsViewController

            destinationVC.selectedStore = selectedStores[0]
        }
    }

Second viewcontroller (StoreDetailsViewController):

var selectedStore = String()

I have another viewcontroller doing the exact same thing, but the array is not empty inside the prepareforsegue function. I have the tableview cell connected to the secondviewcontroller through a segue called "showStoreDetailsSegue".


问:
@PostConstruct and "No Hibernate Session bound to thread" exception
I have to do some database stuff in my repository' @PostConstruct:

```<code>@Repository
public class SomeRepositoryHibernate implements SomeRepository {
    private SessionFactory sessionFactory;

    @Autowired
    public SomeRepositoryHibernate(SessionFactory sessionFactory) {
        this.sessionFactory = sessionFactory;
    }

    ...

    @PostConstruct
    public void doSomestuffWithDb() {
        ...
    }

}
</code>```

but it fails with:

```<code>org.hibernate.HibernateException: No Hibernate Session bound to thread, and
   configuration does not allow creation of non-transactional one here
</code>```

is there any easy solution for that?

Thanks!

答:
You don't have a running transaction in @PostConstruct

you can't use @Transactional there (except for mode="aspectj"), so spring can't start the transaction

a transaction is required for hibernate to mutate data (insert/update/delete)

So, you would have to create your session from the session factory (via .openSession()) there and start a transaction manually.

Body:
I have to do some database stuff in my repository' @PostConstruct:

@Repository
public class SomeRepositoryHibernate implements SomeRepository {
    private SessionFactory sessionFactory;

    @Autowired
    public SomeRepositoryHibernate(SessionFactory sessionFactory) {
        this.sessionFactory = sessionFactory;
    }

    ...

    @PostConstruct
    public void doSomestuffWithDb() {
        ...
    }

}

but it fails with:

org.hibernate.HibernateException: No Hibernate Session bound to thread, and
   configuration does not allow creation of non-transactional one here

is there any easy solution for that?

Thanks!


问:
How to de-structure/match json type in Ballerina?
In order to return a default json in case of an error, I've written the following function where I'm trying to use a match statement against a json|error type in ballerina 0.991.0

This:

```<code>function jsonIt(json|error myJson)returns json{
    match myJson {
        var error(detail,message) => return {};
        _ => return {};
    }
}
</code>```

Compiles (ignore the swallowed error for the sake of argument), but obviously doesn't return the json value.

This:

```<code>function jsonIt(json|error myJson)returns json{
    match myJson {
        json j  => return j;
        var error(detail,reason) => return {};
    }
}
</code>```

Won't compile => extraneous input 'json'

Basically, I can't find any way to match against the possible json type. I've tried:

json mj=> return mj;

var mj => return mj; //doesn't limit type to json

and a few others. Basically, I can't find a way to match the json type, or destructure it similar to error(d,m)

While this questions stands on its own, suggesting a more idiomatic way to accomplish this would also be appreciated.

答:
Based on your problem description, I think you are trying to do a type switch here. i.e., checking whether a value belongs to a particular type. You can use type test expression in Ballerina for this use case. Following two approaches should work.

```<code>function jsonIt(json | error myJson) returns json {
    if (myJson is json) {
       return myJson;
    } else {
       return {};
    }
}
</code>```

With the ternary operator

```<code>function jsonIt(json | error myJson) returns json {
    return (myJson is json) ? myJson :{};
}
</code>```

The match statement performs a value switch. i.e., select the block to execute based on the pattern that a value matches

Body:
In order to return a default json in case of an error, I've written the following function where I'm trying to use a match statement against a json|error type in ballerina 0.991.0

This:

function jsonIt(json|error myJson)returns json{
    match myJson {
        var error(detail,message) => return {};
        _ => return {};
    }
}

Compiles (ignore the swallowed error for the sake of argument), but obviously doesn't return the json value.

This:

function jsonIt(json|error myJson)returns json{
    match myJson {
        json j  => return j;
        var error(detail,reason) => return {};
    }
}

Won't compile => extraneous input 'json'

Basically, I can't find any way to match against the possible json type. I've tried:

json mj=> return mj;

var mj => return mj; //doesn't limit type to json

and a few others. Basically, I can't find a way to match the json type, or destructure it similar to error(d,m)

While this questions stands on its own, suggesting a more idiomatic way to accomplish this would also be appreciated.

Body:
Based on your problem description, I think you are trying to do a type switch here. i.e., checking whether a value belongs to a particular type. You can use type test expression in Ballerina for this use case. Following two approaches should work.

function jsonIt(json | error myJson) returns json {
    if (myJson is json) {
       return myJson;
    } else {
       return {};
    }
}

With the ternary operator

function jsonIt(json | error myJson) returns json {
    return (myJson is json) ? myJson :{};
}

The match statement performs a value switch. i.e., select the block to execute based on the pattern that a value matches


问:
How to use implode a column from an array of stdClass objects?
I have an array of stdClass objects and I want to build a comma separated list using one specific field of all those stdClass objects.  My array looks like this:

```<code>$obj1 = stdClass Object ( [foo] => 4 [bar] => 8 [foo-bar] => 15 );
$obj2 = stdClass Object ( [foo] => 16 [bar] => 23 [foo-bar] => 42 );
$obj3 = stdClass Object ( [foo] => 76 [bar] => 79 [foo-bar] => 83 );

$a = array(1=>$obj1 , 2=>$obj2 , 3=>$obj3);
</code>```

And I want to implode on foo of all the stdClass objects in that array to create a comma separated list.  So the desired result is:

```<code>4,16,76
</code>```

Is there any way to do this with implode (or some other mystery function) without having to put this array of objects through a loop?

答:
With PHP 7.0+ you can use array_column for this.

```<code>echo implode(',', array_column($a, 'foo'));
</code>```

Body:
I have an array of stdClass objects and I want to build a comma separated list using one specific field of all those stdClass objects.  My array looks like this:

$obj1 = stdClass Object ( [foo] => 4 [bar] => 8 [foo-bar] => 15 );
$obj2 = stdClass Object ( [foo] => 16 [bar] => 23 [foo-bar] => 42 );
$obj3 = stdClass Object ( [foo] => 76 [bar] => 79 [foo-bar] => 83 );

$a = array(1=>$obj1 , 2=>$obj2 , 3=>$obj3);

And I want to implode on foo of all the stdClass objects in that array to create a comma separated list.  So the desired result is:

4,16,76

Is there any way to do this with implode (or some other mystery function) without having to put this array of objects through a loop?

Body:
With PHP 7.0+ you can use array_column for this.

echo implode(',', array_column($a, 'foo'));


问:
Is there a way to paint onto a JPanel from an async method?
I am writing a Java based game using Swing and Firebase. I store all my players in a list, and when any of them move the onChildChanged(...) callback (see below) is fired. Since this is async, I obviously can't set any variables that are out of the function's scope. This is was all fine, until I needed to actually paint this player to the screen.

Since I extend a JPanel and overwrite it's paintComponent(Graphics g) I have to do all my drawing within this method (or at least thats what I've been told). The problem is I need to draw this player from inside the callback, and this is what I don't know how to do.

I have tried to "export" the Graphics g object to a global variable, and then use that variable within the async function but this yielded no results. Maybe because it isn't actually drawing to the JPanel's canvas?

So I have a class something like this:

```<code>class Screen extends JPanel {

@Override
public void paintComponent(Graphics g) {
    // All my painting stuff goes here
}

ref.addChildEventListener(new ChildEventListener() {

            // This is the callback
            @Override
            public void onChildChanged(DataSnapshot snapshot, String previousChildName) {
                      // Draw the player
                }
            }
        });

}

</code>```

EDIT: forgot to mention that I am also calling repaint() on every game tick (60 times a second). I don't know if this changes anything but just in case :)

So I want to be able to paint the player either within the callback or paint it in the paintComponent() method, and I'm wondering how this can be done.

Thanks!

答:
You shouldn't paint anywhere except within the paintComponent method. So the solution is:

Update the model fields that are used to tell the GUI what to paint.

If this update will directly affect the GUI in any other way, such as update text fields, labels, JTables and such, then this data change should be performed on the EDT by placing it into the event queue within a Runnable passed into SwingUtilities.invokeLater(...)

Then call repaint()

The GUI will then use those fields and do all painting on the Swing event thread.

Note that repaint() is one of the few Swing methods that can be called off of the EDT.

I am little confused as to what you mean by updating the model's fields though haha, could you maybe clarify that?

If you're creating any complex GUI, you'll likely use a model - view type of structure (such as M-V-C for model-view-controller) where you separate program logic, the model portion, from the GUI, the view. The view depicts the state of the model and allows for user interaction, so I'm talking about state fields.

For example, if you have a game with animation, the model would represent the location of the players, the state of the players (health, weapons, stock), their behaviors, .... while the view would draw all of this based on the model's state.

Body:
I am writing a Java based game using Swing and Firebase. I store all my players in a list, and when any of them move the onChildChanged(...) callback (see below) is fired. Since this is async, I obviously can't set any variables that are out of the function's scope. This is was all fine, until I needed to actually paint this player to the screen.

Since I extend a JPanel and overwrite it's paintComponent(Graphics g) I have to do all my drawing within this method (or at least thats what I've been told). The problem is I need to draw this player from inside the callback, and this is what I don't know how to do.

I have tried to "export" the Graphics g object to a global variable, and then use that variable within the async function but this yielded no results. Maybe because it isn't actually drawing to the JPanel's canvas?

So I have a class something like this:

class Screen extends JPanel {

@Override
public void paintComponent(Graphics g) {
    // All my painting stuff goes here
}

ref.addChildEventListener(new ChildEventListener() {

            // This is the callback
            @Override
            public void onChildChanged(DataSnapshot snapshot, String previousChildName) {
                      // Draw the player
                }
            }
        });

}

EDIT: forgot to mention that I am also calling repaint() on every game tick (60 times a second). I don't know if this changes anything but just in case :)

So I want to be able to paint the player either within the callback or paint it in the paintComponent() method, and I'm wondering how this can be done.

Thanks!


问:
Transposing and Concatenating strings
How do I transpose and concatenate a pandas dataframe without using a for loop?

Here is the input data:

```<code>input_data =  pandas.DataFrame({'a': ['fruit', 'fruit', 'fruit', 'food', 'food', 'food', 'food'],
                      'b': ['banana', '', 'apple', 'rice', '', 'yam', 'chicken']})
</code>```

Resulting output should look like this:

```<code>result = pandas.DataFrame({'a': ['fruit', 'food'],
                      'b': ['banana  apple', 'rice  yam  chicken']})
</code>```

Here is my for loop solution:

```<code>stuff_list = input_data.a.drop_duplicates().tolist()
result = pandas.DataFrame()

for s in stuff_list:
    step1 = input_data[input_data.a == s]
    step2 = ' '.join(step1.b.tolist())
    step3 = pandas.DataFrame({'a':[s], 'b':[step2]})
    result = result.append(step3)

print(result)
</code>```

答:
One way is to group by column a and apply a string join to column b:

```<code>(input_data.groupby('a', sort=False)['b']
           .apply(lambda x: ' '.join(x))
           .reset_index())

       a                  b
0  fruit      banana  apple
1   food  rice  yam chicken
</code>```

Title: Transposing and Concatenating strings

Body:
How do I transpose and concatenate a pandas dataframe without using a for loop?

Here is the input data:

input_data =  pandas.DataFrame({'a': ['fruit', 'fruit', 'fruit', 'food', 'food', 'food', 'food'],
                      'b': ['banana', '', 'apple', 'rice', '', 'yam', 'chicken']})

Resulting output should look like this:

result = pandas.DataFrame({'a': ['fruit', 'food'],
                      'b': ['banana  apple', 'rice  yam  chicken']})

Here is my for loop solution:

stuff_list = input_data.a.drop_duplicates().tolist()
result = pandas.DataFrame()

for s in stuff_list:
    step1 = input_data[input_data.a == s]
    step2 = ' '.join(step1.b.tolist())
    step3 = pandas.DataFrame({'a':[s], 'b':[step2]})
    result = result.append(step3)

print(result)

Body:
One way is to group by column a and apply a string join to column b:

(input_data.groupby('a', sort=False)['b']
           .apply(lambda x: ' '.join(x))
           .reset_index())

       a                  b
0  fruit      banana  apple
1   food  rice  yam chicken