问:
Cythonizing fails because of unknown type name 'uint64_t'
This may be a newbie problem. I can't cythonize a simple helloworld.pyx tutorial script while the exact same code works on linux:

```<code>print("hello world")
</code>```

Here is the setup.py script:

```<code>from distutils.core import setup

from Cython.Build import cythonize

setup(ext_modules = cythonize('helloworld.pyx'))
</code>```

But I get this after running python setup.py build_ext --inplace:

```<code>running build_ext
building 'helloworld' extension
creating build
creating build/temp.macosx-10.7-x86_64-3.6
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/haotian/anaconda3/include -arch x86_64 -I/Users/haotian/anaconda3/include -arch x86_64 -I/Users/haotian/anaconda3/include/python3.6m -c helloworld.c -o build/temp.macosx-10.7-x86_64-3.6/helloworld.o
In file included from helloworld.c:16:
In file included from /Users/haotian/anaconda3/include/python3.6m/Python.h:34:
In file included from /usr/include/stdlib.h:65:
In file included from /usr/include/sys/wait.h:110:
/usr/include/sys/resource.h:196:2: error: unknown type name 'uint8_t'
        uint8_t  ri_uuid[16];
        ^
/usr/include/sys/resource.h:197:2: error: unknown type name 'uint64_t'
        uint64_t ri_user_time;
        ^
/usr/include/sys/resource.h:198:2: error: unknown type name 'uint64_t'
        uint64_t ri_system_time;
        ^
/usr/include/sys/resource.h:199:2: error: unknown type name 'uint64_t'
        uint64_t ri_pkg_idle_wkups;
        ^
/usr/include/sys/resource.h:200:2: error: unknown type name 'uint64_t'
        uint64_t ri_interrupt_wkups;
        ^
/usr/include/sys/resource.h:201:2: error: unknown type name 'uint64_t'
        uint64_t ri_pageins;
        ^
/usr/include/sys/resource.h:202:2: error: unknown type name 'uint64_t'
        uint64_t ri_wired_size;
        ^
/usr/include/sys/resource.h:203:2: error: unknown type name 'uint64_t'
        uint64_t ri_resident_size;
        ^
/usr/include/sys/resource.h:204:2: error: unknown type name 'uint64_t'
        uint64_t ri_phys_footprint;
        ^
/usr/include/sys/resource.h:205:2: error: unknown type name 'uint64_t'
        uint64_t ri_proc_start_abstime;
        ^
/usr/include/sys/resource.h:206:2: error: unknown type name 'uint64_t'
        uint64_t ri_proc_exit_abstime;
        ^
/usr/include/sys/resource.h:210:2: error: unknown type name 'uint8_t'
        uint8_t  ri_uuid[16];
        ^
/usr/include/sys/resource.h:211:2: error: unknown type name 'uint64_t'
        uint64_t ri_user_time;
        ^
/usr/include/sys/resource.h:212:2: error: unknown type name 'uint64_t'
        uint64_t ri_system_time;
        ^
/usr/include/sys/resource.h:213:2: error: unknown type name 'uint64_t'
        uint64_t ri_pkg_idle_wkups;
        ^
/usr/include/sys/resource.h:214:2: error: unknown type name 'uint64_t'
        uint64_t ri_interrupt_wkups;
        ^
/usr/include/sys/resource.h:215:2: error: unknown type name 'uint64_t'
        uint64_t ri_pageins;
        ^
/usr/include/sys/resource.h:216:2: error: unknown type name 'uint64_t'
        uint64_t ri_wired_size;
        ^
/usr/include/sys/resource.h:217:2: error: unknown type name 'uint64_t'
        uint64_t ri_resident_size;
        ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
error: command 'gcc' failed with exit status 1
</code>```

I'm on MacOS 10.13.1, python 3.6.3, Cython 0.27.3 and clang-900.0.38.

Apparently there is a problem with including stdint.h, how should I go about doing it in Cython?

EDIT:
My gcc is in /usr/bin/gcc

```<code>gcc --version
</code>```

shows:

```<code>Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 9.0.0 (clang-900.0.38)
Target: x86_64-apple-darwin17.2.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
</code>```

答:
I have found a solution to my problem.

It seems like after installing High Sierra some header files in the /usr/local/include are messed up, and causing the brew installation to malfunction. Uninstall and reinstalling brew solved the problem.

```<code>rm -rf /usr/local/Cellar /usr/local/.git && brew cleanup
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install )"
</code>```

Body:
This may be a newbie problem. I can't cythonize a simple helloworld.pyx tutorial script while the exact same code works on linux:

print("hello world")

Here is the setup.py script:

from distutils.core import setup

from Cython.Build import cythonize

setup(ext_modules = cythonize('helloworld.pyx'))

But I get this after running python setup.py build_ext --inplace:

running build_ext
building 'helloworld' extension
creating build
creating build/temp.macosx-10.7-x86_64-3.6
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/haotian/anaconda3/include -arch x86_64 -I/Users/haotian/anaconda3/include -arch x86_64 -I/Users/haotian/anaconda3/include/python3.6m -c helloworld.c -o build/temp.macosx-10.7-x86_64-3.6/helloworld.o
In file included from helloworld.c:16:
In file included from /Users/haotian/anaconda3/include/python3.6m/Python.h:34:
In file included from /usr/include/stdlib.h:65:
In file included from /usr/include/sys/wait.h:110:
/usr/include/sys/resource.h:196:2: error: unknown type name 'uint8_t'
        uint8_t  ri_uuid[16];
        ^
/usr/include/sys/resource.h:197:2: error: unknown type name 'uint64_t'
        uint64_t ri_user_time;
        ^
/usr/include/sys/resource.h:198:2: error: unknown type name 'uint64_t'
        uint64_t ri_system_time;
        ^
/usr/include/sys/resource.h:199:2: error: unknown type name 'uint64_t'
        uint64_t ri_pkg_idle_wkups;
        ^
/usr/include/sys/resource.h:200:2: error: unknown type name 'uint64_t'
        uint64_t ri_interrupt_wkups;
        ^
/usr/include/sys/resource.h:201:2: error: unknown type name 'uint64_t'
        uint64_t ri_pageins;
        ^
/usr/include/sys/resource.h:202:2: error: unknown type name 'uint64_t'
        uint64_t ri_wired_size;
        ^
/usr/include/sys/resource.h:203:2: error: unknown type name 'uint64_t'
        uint64_t ri_resident_size;
        ^
/usr/include/sys/resource.h:204:2: error: unknown type name 'uint64_t'
        uint64_t ri_phys_footprint;
        ^
/usr/include/sys/resource.h:205:2: error: unknown type name 'uint64_t'
        uint64_t ri_proc_start_abstime;
        ^
/usr/include/sys/resource.h:206:2: error: unknown type name 'uint64_t'
        uint64_t ri_proc_exit_abstime;
        ^
/usr/include/sys/resource.h:210:2: error: unknown type name 'uint8_t'
        uint8_t  ri_uuid[16];
        ^
/usr/include/sys/resource.h:211:2: error: unknown type name 'uint64_t'
        uint64_t ri_user_time;
        ^
/usr/include/sys/resource.h:212:2: error: unknown type name 'uint64_t'
        uint64_t ri_system_time;
        ^
/usr/include/sys/resource.h:213:2: error: unknown type name 'uint64_t'
        uint64_t ri_pkg_idle_wkups;
        ^
/usr/include/sys/resource.h:214:2: error: unknown type name 'uint64_t'
        uint64_t ri_interrupt_wkups;
        ^
/usr/include/sys/resource.h:215:2: error: unknown type name 'uint64_t'
        uint64_t ri_pageins;
        ^
/usr/include/sys/resource.h:216:2: error: unknown type name 'uint64_t'
        uint64_t ri_wired_size;
        ^
/usr/include/sys/resource.h:217:2: error: unknown type name 'uint64_t'
        uint64_t ri_resident_size;
        ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
error: command 'gcc' failed with exit status 1

I'm on MacOS 10.13.1, python 3.6.3, Cython 0.27.3 and clang-900.0.38.

Apparently there is a problem with including stdint.h, how should I go about doing it in Cython?

EDIT:
My gcc is in /usr/bin/gcc

gcc --version

shows:

Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 9.0.0 (clang-900.0.38)
Target: x86_64-apple-darwin17.2.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Body:
I have found a solution to my problem.

It seems like after installing High Sierra some header files in the /usr/local/include are messed up, and causing the brew installation to malfunction. Uninstall and reinstalling brew solved the problem.

rm -rf /usr/local/Cellar /usr/local/.git && brew cleanup
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install )"


来源: stackexchange

问:
How can I show the most recent events per user with Keen IO?
Suppose you have a Keen IO collection called "survey-completed" that contains events matching the following pattern:

```<code>keen.id: <unique autogenerated id>
keen.timestamp: <autogenerated overridable timestamp>
userId: <hex string for user>
surveyScore: <integer from 1 to 10>
...
</code>```

How would you create a report of only the most up-to-date satisfaction score for each user that responded to one or more surveys within a given amount of time (such as one week)?

答:
There isn't a really elegant way to make it happen, but for a given userId you could successfully return your the most up-to-date event create a count query with a group_by on [surveyScore, keen.timestamp] and an order_by on the keen.timestamp property. You will want to set limit=1 to select only the most recent surveyScore.

If you'd like to use an extraction, the most straight forward way would be to run an extraction with property_names set to ["userId","keen.timestamp","surveyScore"]. Once you receive the results you can then do some client-side post processing. This is probably the best way if you want to take a look at all of your userIds.

If you're interested in a given userId and want to use an extraction, you can run an extraction with a filter on the userId eq X, define the optional parameter latest set to latest=1. The latest property is an integer containing the number of most recent events to extract. Note: The use of latest will call upon the keen.created_at timestamp instead of keen.timestamp (https://keen.io/docs/api/#the-keen-object).

ContentLicense: CC BY-SA 3.0

Body:
Suppose you have a Keen IO collection called "survey-completed" that contains events matching the following pattern:

keen.id: <unique autogenerated id>
keen.timestamp: <autogenerated overridable timestamp>
userId: <hex string for user>
surveyScore: <integer from 1 to 10>
...

How would you create a report of only the most up-to-date satisfaction score for each user that responded to one or more surveys within a given amount of time (such as one week)?


问:
pandas drop values in a columns based other columns
Is there a way to drop a single entry in a df?

```<code>df1:
ID  Count1  Value1  Count2  Value2
111 5   10  5   25
112 4   15  6   25
113 6   7   4   10
114 3   8   2   1
115 7   1   8   10
</code>```

I want to drop any Value1 if Count1 < 5 and drop any Value2 if Count2 < 5.  The result would be like this:

```<code>ID  Count1  Value1  Count2  Value2
111 5   10  5   25
112 NA  NA  6   25
113 6   7   4   NA
114 NA  NA  NA  NA
115 7   1   8   10
</code>```

Thanks!

答:
Use mask:

```<code>df[['Count1','Value1']] = df[['Count1','Value1']].mask(df.Count1 < 5)
df[['Count2','Value2']] = df[['Count2','Value2']].mask(df.Count2 < 5)

print (df)
    ID  Count1  Value1  Count2  Value2
0  111     5.0    10.0     5.0    25.0
1  112     NaN     NaN     6.0    25.0
2  113     6.0     7.0     NaN     NaN
3  114     NaN     NaN     NaN     NaN
4  115     7.0     1.0     8.0    10.0
</code>```

Body:
Is there a way to drop a single entry in a df?

df1:
ID  Count1  Value1  Count2  Value2
111 5   10  5   25
112 4   15  6   25
113 6   7   4   10
114 3   8   2   1
115 7   1   8   10

I want to drop any Value1 if Count1 < 5 and drop any Value2 if Count2 < 5.  The result would be like this:

ID  Count1  Value1  Count2  Value2
111 5   10  5   25
112 NA  NA  6   25
113 6   7   4   NA
114 NA  NA  NA  NA
115 7   1   8   10

Thanks!

Body:
Use mask:

df[['Count1','Value1']] = df[['Count1','Value1']].mask(df.Count1 < 5)
df[['Count2','Value2']] = df[['Count2','Value2']].mask(df.Count2 < 5)

print (df)
    ID  Count1  Value1  Count2  Value2
0  111     5.0    10.0     5.0    25.0
1  112     NaN     NaN     6.0    25.0
2  113     6.0     7.0     NaN     NaN
3  114     NaN     NaN     NaN     NaN
4  115     7.0     1.0     8.0    10.0


问:
plot count of discrete data by date
I am new to ggplot2 and trying to plot a continuous histogram showing the evolution of reviews by date and rating.

My data set look like this:

```<code>        date rating reviews
1 2017-11-24      1 some text here
2 2017-11-24      1 some text here
3 2017-12-02      5 some text here
4 2017-11-24      3 some text here
5 2017-11-24      3 some text here
6 2017-11-24      4 some text here
</code>```

What I want to get is something like this:

for rating == 1

```<code>        date    count
1  2017-11-24      2
2  2017-11-25      7
.
.
.
</code>```

and so on for rating == 2 and 3

I've tried

```<code>ggplot(aes(x = date, y = rating), data = df) + geom_line()
</code>```

but it gives me only rating on the y axis and not counts:

答:
You can use dplyr to get the desired dataset and pass that into ggplot();

```<code>library(dplyr)
library(ggplot2)

 sample_data %>% group_by(rating,date) %>% summarise(n=n()) %>%
                ggplot(aes(x=date, y=n, group=rating, color=as.factor(rating))) +
                          geom_line(size=1.5) + geom_point()
</code>```

Data:

```<code>sample_data <- structure(list(id = c(1L, 2L, 2L, 3L, 4L, 5L, 5L, 6L, 6L, 1L,
     2L, 3L, 3L, 4L, 5L, 6L, 1L, 2L, 2L, 2L, 3L, 4L, 5L, 6L), date = structure(c(1L,
     1L, 3L, 7L, 1L, 1L, 1L, 1L, 5L, 2L, 3L, 8L, 8L, 3L, 4L, 5L, 5L,
     6L, 6L, 6L, 9L, 6L, 6L, 6L), .Label = c("2017-11-24", "2017-11-25",
     "2017-11-26", "2017-11-27", "2017-11-28", "2017-11-29", "2017-12-02",
     "2017-12-04", "2017-12-08"), class = "factor"), rating = c(1L,
     1L, 1L, 5L, 3L, 3L, 3L, 4L, 4L, 1L, 1L, 5L, 5L, 3L, 3L, 4L, 1L,
     1L, 1L, 1L, 5L, 3L, 3L, 4L), reviews = structure(c(1L, 1L, 1L,
     1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
     1L, 1L, 1L, 1L, 1L), .Label = "review", class = "factor")), .Names = c("id",
     "date", "rating", "reviews"), row.names = c(NA, 24L), class = "data.frame")
</code>```

Title: plot count of discrete data by date

Body:
I am new to ggplot2 and trying to plot a continuous histogram showing the evolution of reviews by date and rating.

My data set look like this:

        date rating reviews
1 2017-11-24      1 some text here
2 2017-11-24      1 some text here
3 2017-12-02      5 some text here
4 2017-11-24      3 some text here
5 2017-11-24      3 some text here
6 2017-11-24      4 some text here

What I want to get is something like this:

for rating == 1

        date    count
1  2017-11-24      2
2  2017-11-25      7
.
.
.

and so on for rating == 2 and 3

I've tried

ggplot(aes(x = date, y = rating), data = df) + geom_line()

but it gives me only rating on the y axis and not counts:

Body:
You can use dplyr to get the desired dataset and pass that into ggplot();

library(dplyr)
library(ggplot2)

 sample_data %>% group_by(rating,date) %>% summarise(n=n()) %>%
                ggplot(aes(x=date, y=n, group=rating, color=as.factor(rating))) +
                          geom_line(size=1.5) + geom_point()

Data:

sample_data <- structure(list(id = c(1L, 2L, 2L, 3L, 4L, 5L, 5L, 6L, 6L, 1L,
     2L, 3L, 3L, 4L, 5L, 6L, 1L, 2L, 2L, 2L, 3L, 4L, 5L, 6L), date = structure(c(1L,
     1L, 3L, 7L, 1L, 1L, 1L, 1L, 5L, 2L, 3L, 8L, 8L, 3L, 4L, 5L, 5L,
     6L, 6L, 6L, 9L, 6L, 6L, 6L), .Label = c("2017-11-24", "2017-11-25",
     "2017-11-26", "2017-11-27", "2017-11-28", "2017-11-29", "2017-12-02",
     "2017-12-04", "2017-12-08"), class = "factor"), rating = c(1L,
     1L, 1L, 5L, 3L, 3L, 3L, 4L, 4L, 1L, 1L, 5L, 5L, 3L, 3L, 4L, 1L,
     1L, 1L, 1L, 5L, 3L, 3L, 4L), reviews = structure(c(1L, 1L, 1L,
     1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
     1L, 1L, 1L, 1L, 1L), .Label = "review", class = "factor")), .Names = c("id",
     "date", "rating", "reviews"), row.names = c(NA, 24L), class = "data.frame")


问:
Document storage in phpmyadmin vs website folder?
What is the best way to store documents that saves space? By letting user documents to be stored in phpmyadmin db or by creating folders and letting user upload documents there?

I want to allow scans like pdf and images and also word docs and .txt files.

Also, how will i go about uploading documents? Any tutorials i can use?  New to this.

Thanks!

答:
phpmyadmin, is an script that lets you edit\add\etc mysql db's its not a db. Depending on the type of documents, and what you plan to do with them I would always say store them in the file system, you can store there name\path in the db.

file upload tutorial: Handling file uploads

ContentLicense: CC BY-SA 2.5


问:
Javascript: Loop Through Object By Number Index Rather Than Key-Value
While looping through an object is simple using for(key in object), I'd like to access an object via an index (like an array) rather than its value.

I have a "days of the week" object composed of:

```<code>days: { sunday: "N", monday: "Y", tuesday: "N", wednesday: "Y",
thursday: "N", friday: "Y", saturday: "N" }
</code>```

I want to use a for-loop which cycles through the seven days of the week (0-6) and checks whether the object is "Y" or "N" for that day (checking the "days" object for the value at key 0 (sunday), 1 (monday), 2, 3, etc.)

```<code>for (var i = 0; i < 7; i++) { }
</code>```

My problem could be solved with a bunch of if statements if i == 0 { //check sunday } else if i == 1 { //check monday } or manipulating my object into multiple arrays and going from there, however neither of these are very elegant.  Is there a way to loop through the days object and access each value by a key (0 for first, 1 for second, etc.)?

答:
To iterate over the properties of an object in a consistent order,
you have no choice but to use an array of the property names,
for example:

```<code>var daysList = [ 'sunday', 'monday', 'tuesday', 'wednesday', 'thursday', 'friday', 'saturday' ];
</code>```

And then you can iterate over the array of names:

```<code>for (let i = 0; i < daysList.length; i++) {
    let day = daysList[i];
    // do what you need with days[day]
}
</code>```

Body:
While looping through an object is simple using for(key in object), I'd like to access an object via an index (like an array) rather than its value.

I have a "days of the week" object composed of:

days: { sunday: "N", monday: "Y", tuesday: "N", wednesday: "Y",
thursday: "N", friday: "Y", saturday: "N" }

I want to use a for-loop which cycles through the seven days of the week (0-6) and checks whether the object is "Y" or "N" for that day (checking the "days" object for the value at key 0 (sunday), 1 (monday), 2, 3, etc.)

for (var i = 0; i < 7; i++) { }

My problem could be solved with a bunch of if statements if i == 0 { //check sunday } else if i == 1 { //check monday } or manipulating my object into multiple arrays and going from there, however neither of these are very elegant.  Is there a way to loop through the days object and access each value by a key (0 for first, 1 for second, etc.)?

Body:
To iterate over the properties of an object in a consistent order,
you have no choice but to use an array of the property names,
for example:

var daysList = [ 'sunday', 'monday', 'tuesday', 'wednesday', 'thursday', 'friday', 'saturday' ];

And then you can iterate over the array of names:

for (let i = 0; i < daysList.length; i++) {
    let day = daysList[i];
    // do what you need with days[day]
}


问:
amazon emr spark submission from S3 not working
I have a cluster up and running. I am trying to add a step to run my code. The code itself works fine on a single instance. Only thing is I can't get it to work off S3.

```<code>aws emr add-steps --cluster-id j-XXXXX --steps Type=spark,Name=SomeSparkApp,Args=[--deploy-mode,cluster,--executor-memory,0.5g,s3://<mybucketname>/mypythonfile.py]
</code>```

This is exactly what examples show I should do. What am I doing wrong?

Error I get:

```<code>Exception in thread "main" java.lang.IllegalArgumentException: Unknown/unsupported param List(--executor-memory, 0.5g, --executor-cores, 2, --primary-py-file, s3://<mybucketname>/mypythonfile.py, --class, org.apache.spark.deploy.PythonRunner)

Usage: org.apache.spark.deploy.yarn.Client [options]
Options:
  --jar JAR_PATH           Path to your application's JAR file (required in yarn-cluster
                           mode)
    .
    .
    .
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Command exiting with ret '1'
</code>```

When I specify as this instead:

```<code>aws emr add-steps --cluster-id j-XXXXX --steps Type=spark,Name= SomeSparkApp,Args=[--executor-memory,0.5g,s3://<mybucketname>/mypythonfile.py]
</code>```

I get this error instead:

```<code>Error: Only local python files are supported: Parsed arguments:
  master                  yarn-client
  deployMode              client
  executorMemory          0.5g
  executorCores           2
</code>```

EDIT: IT gets further along when I manually create the python file after SSH'ing into the cluster, and specifying as follows:

```<code>aws emr add-steps --cluster-id 'j-XXXXX' --steps Type=spark,Name= SomeSparkApp,Args=[--executor-memory,1g,/home/hadoop/mypythonfile.py]
</code>```

But, not doing the job.

Any help appreciated. This is really frustrating as a well documented method on AWS's own blog here https://blogs.aws.amazon.com/bigdata/post/Tx578UTQUV7LRP/Submitting-User-Applications-with-spark-submit does not work.

答:
I searched for days and finally discovered this thread which states

PySpark currently only supports local
  files. This does not mean it only runs in local mode, however; you can
  still run PySpark on any cluster manager (though only in client mode). All
  this means is that your python files must be on your local file system.
  Until this is supported, the straightforward workaround then is to just
  copy the files to your local machine.

Body:
I have a cluster up and running. I am trying to add a step to run my code. The code itself works fine on a single instance. Only thing is I can't get it to work off S3.

aws emr add-steps --cluster-id j-XXXXX --steps Type=spark,Name=SomeSparkApp,Args=[--deploy-mode,cluster,--executor-memory,0.5g,s3://<mybucketname>/mypythonfile.py]

This is exactly what examples show I should do. What am I doing wrong?

Error I get:

Exception in thread "main" java.lang.IllegalArgumentException: Unknown/unsupported param List(--executor-memory, 0.5g, --executor-cores, 2, --primary-py-file, s3://<mybucketname>/mypythonfile.py, --class, org.apache.spark.deploy.PythonRunner)

Usage: org.apache.spark.deploy.yarn.Client [options]
Options:
  --jar JAR_PATH           Path to your application's JAR file (required in yarn-cluster
                           mode)
    .
    .
    .
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Command exiting with ret '1'

When I specify as this instead:

aws emr add-steps --cluster-id j-XXXXX --steps Type=spark,Name= SomeSparkApp,Args=[--executor-memory,0.5g,s3://<mybucketname>/mypythonfile.py]

I get this error instead:

Error: Only local python files are supported: Parsed arguments:
  master                  yarn-client
  deployMode              client
  executorMemory          0.5g
  executorCores           2

EDIT: IT gets further along when I manually create the python file after SSH'ing into the cluster, and specifying as follows:

aws emr add-steps --cluster-id 'j-XXXXX' --steps Type=spark,Name= SomeSparkApp,Args=[--executor-memory,1g,/home/hadoop/mypythonfile.py]

But, not doing the job.

Any help appreciated. This is really frustrating as a well documented method on AWS's own blog here https://blogs.aws.amazon.com/bigdata/post/Tx578UTQUV7LRP/Submitting-User-Applications-with-spark-submit does not work.


问:
NgRX store select returns store state instead of data
I'm trying to implement ngrx v4 to my angular project and struggling with getting data from ngrx store. I wrote Actions, Reducer and Effects based on provided api documentation on github page and it seems to me everything is correct but instead of data I get store State like this:

So I appreciate if somebody could help me to figure out where my mistake is. Thanks in advance. My code below

interface.ts

```<code>export interface Category {
  id?: number;
  name: string;
  used: boolean;
}
</code>```

actions.ts

```<code>import { Action } from '@ngrx/store';
import {Category} from '../categories';

export const GET_CATEGORIES  = '[Category] Get data';
export const GET_CATEGORIES_ERROR = '[Category] Get data error';
export const GET_CATEGORIES_SUCCESS  = '[Category] Get data success ';

export class GetCategories implements Action {
  readonly type = GET_CATEGORIES;
}
export class GetCategoriesError implements Action {
  readonly type = GET_CATEGORIES_ERROR;
}
export class GetCategoriesSuccess implements Action {
  readonly type = GET_CATEGORIES_SUCCESS;
  constructor(public payload: Category[]) { }
}

export type All = GetCategories | GetCategoriesSuccess | GetCategoriesError
</code>```

reducer.ts

```<code>import * as CategoriesActions from './categories.actions';
import {Category} from '../categories';

export type Action = CategoriesActions.All;

export function categoriesReducer (state: Category[], action: Action) {

  switch (action.type) {

    case CategoriesActions.GET_CATEGORIES: {
      return state;
    }

    case CategoriesActions.GET_CATEGORIES_ERROR: {
      return state;
    }

    case CategoriesActions.GET_CATEGORIES_SUCCESS: {
      return action.payload;
    }

    default: {
      return state;
    }
  }
}
</code>```

effects.ts

```<code>...
import {CategoriesService} from './categories.service';

@Injectable()
export class CategoriesEffects {

  @Effect() getCategories$: Observable<Action> = this.actions$
    .ofType('GET_CATEGORIES')
    .mergeMap(action => this.categoriesService.getCategories()
      .map(data => ({type: 'GET_CATEGORIES_SUCCESS', payload: data }))
      .catch(() => of({ type: 'GET_CATEGORIES_ERROR', payload: {message: 'Oops something is wrong'} }))
    );

  constructor(
    private actions$: Actions,
    private categoriesService: CategoriesService
  ) {}
}
</code>```

component.ts

```<code>interface CategoryState {
  categories: Category[]
}

@Component({
  selector: 'app-categories',
  templateUrl: './categories.component.html',
  styleUrls: ['./categories.component.css'],
})

export class CategoriesComponent implements OnInit {

  categories$: Observable<Category[]>;

  constructor(private store: Store<CategoryState>) {

    console.log(this.categories$) // undefined

    this.categories$ = this.store.select('categories');

    console.log(this.categories$) // store state
  }

  ngOnInit() {
    this.getCategories()
  }

  getCategories () {
    this.store.dispatch({ type: 'GET_CATEGORIES' });
  }
}
</code>```

module

```<code>@NgModule({
  imports: [
    ...
    StoreModule.forRoot({ categories: categoriesReducer }),
    EffectsModule.forRoot([CategoriesEffects]),
  ],
  declarations: [ CategoriesComponent ],
  providers:    [ CategoriesService ],
  entryComponents: []
})
export class CategoriesModule {}
</code>```

答:
Since you are using ngrx/store : 4.1

You should have a reducer factory to inject the reducers to the StoreModule

```<code>import {ActionReducerMap} from '@ngrx/store';
import {Category} from '../categories';
import * as categoryReducer from './categories.reducer';

export interface CategoryState {
  categories: Category[]
}

export interface AppStates {
  categoryState: CategoryState;
}

export const categoryReducers: ActionReducerMap<AppStates> = {
  categoryState: categoryReducer.reducer
};
</code>```

Use the reducer factory to inject into the module as below,

```<code>StoreModule.forRoot(categoryReducers)
import {categoryReducers} from './store/categories.reducer.factory';
</code>```

Your constructor should take the AppStates as the type for Store as

```<code>constructor(private store: Store<AppStates>){}
</code>```

And your effect should use be

```<code>@Effect()
getCategories$: Observable<Action> = this.actions$
    .ofType(CategoriesActions.GET_CATEGORIES)
    .mergeMap(action => this.categoriesService.getCategories()
          .map(data => ({type: CategoriesActions.GET_CATEGORIES_SUCCESS, payload: data }))
          .catch(() => of({ type: CategoriesActions.GET_CATEGORIES_ERROR, payload: {message: 'Oops something is wrong'} }))
    );
</code>```

Body:
I'm trying to implement ngrx v4 to my angular project and struggling with getting data from ngrx store. I wrote Actions, Reducer and Effects based on provided api documentation on github page and it seems to me everything is correct but instead of data I get store State like this:

So I appreciate if somebody could help me to figure out where my mistake is. Thanks in advance. My code below

interface.ts

export interface Category {
  id?: number;
  name: string;
  used: boolean;
}

actions.ts

import { Action } from '@ngrx/store';
import {Category} from '../categories';

export const GET_CATEGORIES  = '[Category] Get data';
export const GET_CATEGORIES_ERROR = '[Category] Get data error';
export const GET_CATEGORIES_SUCCESS  = '[Category] Get data success ';

export class GetCategories implements Action {
  readonly type = GET_CATEGORIES;
}
export class GetCategoriesError implements Action {
  readonly type = GET_CATEGORIES_ERROR;
}
export class GetCategoriesSuccess implements Action {
  readonly type = GET_CATEGORIES_SUCCESS;
  constructor(public payload: Category[]) { }
}

export type All = GetCategories | GetCategoriesSuccess | GetCategoriesError

reducer.ts

import * as CategoriesActions from './categories.actions';
import {Category} from '../categories';

export type Action = CategoriesActions.All;

export function categoriesReducer (state: Category[], action: Action) {

  switch (action.type) {

    case CategoriesActions.GET_CATEGORIES: {
      return state;
    }

    case CategoriesActions.GET_CATEGORIES_ERROR: {
      return state;
    }

    case CategoriesActions.GET_CATEGORIES_SUCCESS: {
      return action.payload;
    }

    default: {
      return state;
    }
  }
}

effects.ts

...
import {CategoriesService} from './categories.service';

@Injectable()
export class CategoriesEffects {

  @Effect() getCategories$: Observable<Action> = this.actions$
    .ofType('GET_CATEGORIES')
    .mergeMap(action => this.categoriesService.getCategories()
      .map(data => ({type: 'GET_CATEGORIES_SUCCESS', payload: data }))
      .catch(() => of({ type: 'GET_CATEGORIES_ERROR', payload: {message: 'Oops something is wrong'} }))
    );

  constructor(
    private actions$: Actions,
    private categoriesService: CategoriesService
  ) {}
}

component.ts

interface CategoryState {
  categories: Category[]
}

@Component({
  selector: 'app-categories',
  templateUrl: './categories.component.html',
  styleUrls: ['./categories.component.css'],
})

export class CategoriesComponent implements OnInit {

  categories$: Observable<Category[]>;

  constructor(private store: Store<CategoryState>) {

    console.log(this.categories$) // undefined

    this.categories$ = this.store.select('categories');

    console.log(this.categories$) // store state
  }

  ngOnInit() {
    this.getCategories()
  }

  getCategories () {
    this.store.dispatch({ type: 'GET_CATEGORIES' });
  }
}

module

@NgModule({
  imports: [
    ...
    StoreModule.forRoot({ categories: categoriesReducer }),
    EffectsModule.forRoot([CategoriesEffects]),
  ],
  declarations: [ CategoriesComponent ],
  providers:    [ CategoriesService ],
  entryComponents: []
})
export class CategoriesModule {}

Body:
Since you are using ngrx/store : 4.1

You should have a reducer factory to inject the reducers to the StoreModule

import {ActionReducerMap} from '@ngrx/store';
import {Category} from '../categories';
import * as categoryReducer from './categories.reducer';

export interface CategoryState {
  categories: Category[]
}

export interface AppStates {
  categoryState: CategoryState;
}

export const categoryReducers: ActionReducerMap<AppStates> = {
  categoryState: categoryReducer.reducer
};

Use the reducer factory to inject into the module as below,

StoreModule.forRoot(categoryReducers)
import {categoryReducers} from './store/categories.reducer.factory';

Your constructor should take the AppStates as the type for Store as

constructor(private store: Store<AppStates>){}

And your effect should use be

@Effect()
getCategories$: Observable<Action> = this.actions$
    .ofType(CategoriesActions.GET_CATEGORIES)
    .mergeMap(action => this.categoriesService.getCategories()
          .map(data => ({type: CategoriesActions.GET_CATEGORIES_SUCCESS, payload: data }))
          .catch(() => of({ type: CategoriesActions.GET_CATEGORIES_ERROR, payload: {message: 'Oops something is wrong'} }))
    );


问:
Send command over a mapped port via kubernetes service
I have an application that can receive commands from a specific port like so:

```<code>echo <command> | nc <hostname> <port>
</code>```

In this case it is opening port 22082, I believe in it's Docker container.

When I place this application into a kubernetes pod, I need to expose it by creating a kubernetes service. Here is my service:

```<code> apiVersion: v1
 kind: Service
 metadata:
  name: commander
 spec:
   selector:
     app: commander
   ports:
     - protocol: TCP
       port: 22282
       targetPort: 22082
   #type: NodePort
   externalIPs:
   -  10.10.30.19
</code>```

NOTE: I commented out NodePort because I haven't been able to expose the port using that method. Whenever I use sudo netstat -nlp | grep 22282 I get nothing.

Using an external IP i'm able to find the port and connect to it using netcat, but whenever I issue a command over the port, it just hangs.

Normally I should be able to issue a 'help' command and get information on the app. With kubernetes I can't get that same output.

Now, if I use hostNetwork: true in my app yaml (not the service), I can connect to the port and get my 'help' info.

What could be keeping my command from reaching the app while not using hostNetwork configuration?

Thanks

UPDATE: Noticed this message from sudo iptables --list:

```<code>Chain KUBE-SERVICES (1 references)
target     prot opt source               destination
REJECT     tcp  --  anywhere             172.21.155.23        /* default/commander: has no endpoints */ tcp dpt:22282 reject-with icmp-port-unreachable
</code>```

UPDATE #2: I solved the above error by setting spec.template.metadata.labels.app to commander. I still, however, am experiencing an inability to send any command to the app.

答:
Thanks to @sfgroups I discovered that I needed to set an actual nodePort like so:

```<code>apiVersion: v1
 kind: Service
 metadata:
  name: commander
 spec:
   selector:
     app: commander
   ports:
     - protocol: TCP
       port: 22282
       nodePort: 32282
       targetPort: 22082
   type: NodePort
</code>```

Pretty odd behavior, makes me wonder what the point of the port field even is!

Body:
I have an application that can receive commands from a specific port like so:

echo <command> | nc <hostname> <port>

In this case it is opening port 22082, I believe in it's Docker container.

When I place this application into a kubernetes pod, I need to expose it by creating a kubernetes service. Here is my service:

 apiVersion: v1
 kind: Service
 metadata:
  name: commander
 spec:
   selector:
     app: commander
   ports:
     - protocol: TCP
       port: 22282
       targetPort: 22082
   #type: NodePort
   externalIPs:
   -  10.10.30.19

NOTE: I commented out NodePort because I haven't been able to expose the port using that method. Whenever I use sudo netstat -nlp | grep 22282 I get nothing.

Using an external IP i'm able to find the port and connect to it using netcat, but whenever I issue a command over the port, it just hangs.

Normally I should be able to issue a 'help' command and get information on the app. With kubernetes I can't get that same output.

Now, if I use hostNetwork: true in my app yaml (not the service), I can connect to the port and get my 'help' info.

What could be keeping my command from reaching the app while not using hostNetwork configuration?

Thanks

UPDATE: Noticed this message from sudo iptables --list:

Chain KUBE-SERVICES (1 references)
target     prot opt source               destination
REJECT     tcp  --  anywhere             172.21.155.23        /* default/commander: has no endpoints */ tcp dpt:22282 reject-with icmp-port-unreachable

UPDATE #2: I solved the above error by setting spec.template.metadata.labels.app to commander. I still, however, am experiencing an inability to send any command to the app.

Body:
Thanks to @sfgroups I discovered that I needed to set an actual nodePort like so:

apiVersion: v1
 kind: Service
 metadata:
  name: commander
 spec:
   selector:
     app: commander
   ports:
     - protocol: TCP
       port: 22282
       nodePort: 32282
       targetPort: 22082
   type: NodePort

Pretty odd behavior, makes me wonder what the point of the port field even is!


问:
How to combine multiple Expression<Func<T, bool>> into a single expression to execute against DbContext?
I'm trying to create a way to allow users of my app to create their own filter views of data. To do that I'm composing a list of Key-Operator-Value pairs which I then try, after multiple steps, to parse into an expression to use with a Where call. So far, I can get to a point where I have the expressions created, but I can't figure out how to merge them into a single expression.

Could someone point me in the right direction on how to do so? From reading similar posts online, I've settled on trying to do it using the LINQKit PredicateBuilder, but all I'm getting back is the initial starting predicate. Here's the code I have so far, it is by no means final, I'm just prototyping it in LINQPad:

```<code>void Main() {
    //  1. Create form post object
    var query = new QueryEdit {
        Filters = new List<Filter> {
            new Filter {
                Id = 1,
                Key = "Name",
                Operator = "=",
                Value = "New York"
            },
            new Filter {
                Id = 2,
                Key = "Name",
                Operator = "!=",
                Value = "Boston"
            },
            new Filter {
                Id = 3,
                Key = "Name",
                Operator = "=",
                Value = "Washington"
            }
        },
        FilterClause = "1 AND 3 OR 2 OR 4"
    };

    //  2. Compose filter groups
    var filterGroups = GetFilterGroups(query);

    //  3. Compose expression groups
    var expressionGroups = GetExpressionGroups<Region>(filterGroups);

    expressionGroups.Dump();

    //  4. Compose full expression
    var expression = GetExpression(expressionGroups);

    expression.Dump();

    //  5. Serialize expression to blob
    //  6. Deserialize blob to expression
    //  7. Execute expression against db
}

static IEnumerable<FilterGroup> GetFilterGroups(
    QueryEdit query) {
    //  The parentheses are meaningless, so they need to be removed.
    //  The groups are all conditions with ANDs, separated by ORs.

    return query.FilterClause.Replace(")", string.Empty).Replace("(", string.Empty).Split(new[] { "OR" }, StringSplitOptions.None).Select(
        fc => new FilterGroup {
            Filters = fc.Split(new[] { "AND" }, StringSplitOptions.None).Select(
                i => {
                    var id = Convert.ToByte(i);

                    return query.Filters.SingleOrDefault(
                        f => f.Id == id);
                }).Where(
                f => f != null).OrderBy(
                f => f.Id)
        }).Where(
        fg => fg.Filters.Any());
}

static IEnumerable<IEnumerable<Expression<Func<T, bool>>>> GetExpressionGroups<T>(
    IEnumerable<FilterGroup> filterGroups) {
    var type = typeof(T);
    var parameter = Expression.Parameter(type);

    return filterGroups.Select(
        fg => {
            return fg.Filters.Select(
                f => {
                    var left = Expression.Property(parameter, type.GetProperty(f.Key));
                    var right = Expression.Constant(f.Value);
                    var body = GetExpressionBody(f.Operator, left, right);

                    if (body == null) {
                        return null;
                    }

                    return Expression.Lambda<Func<T, bool>>(body, parameter);
                }).Where(
                e => e != null);
        });
}

static object GetExpression<T>(
    IEnumerable<IEnumerable<Expression<Func<T, bool>>>> expressionGroups) {
    var predicate = PredicateBuilder.True<T>();

    foreach (var expressionGroup in expressionGroups) {
        var expression = GetPredicateForGroup<T>(expressionGroup);

        predicate.Or(expression);
    }

    return predicate;
}

static Expression<Func<T, bool>> GetPredicateForGroup<T>(
    IEnumerable<Expression<Func<T, bool>>> expressionGroup) {
    var predicate = PredicateBuilder.True<T>();

    foreach (var expression in expressionGroup) {
        predicate.And(expression);
    }

    return predicate;
}

static Expression GetExpressionBody(
    string @operator,
    Expression left,
    Expression right) {
    switch (@operator) {
        case "=":
            return Expression.Equal(left, right);
        case "!=":
            return Expression.NotEqual(left, right);
        case ">":
            return Expression.GreaterThan(left, right);
        case ">=":
            return Expression.GreaterThanOrEqual(left, right);
        case "<":
            return Expression.LessThan(left, right);
        case "<=":
            return Expression.LessThanOrEqual(left, right);
        default:
            return null;
    }
}

sealed class QueryEdit {
    public IEnumerable<Filter> Filters { get; set; }
    public string FilterClause { get; set; }
    public int Id { get; set; }
    public string Name { get; set; }
}

sealed class Filter {
    public byte Id { get; set; }
    public string Key { get; set; }
    public string Operator { get; set; }
    public object Value { get; set; }
}

sealed class FilterGroup {
    public IEnumerable<Filter> Filters { get; set; }
}
</code>```

I'm also open to other suggestions on how to accomplish this, or overall improvements. Thanks in advance!

答: You could look at using the Specification pattern to combine predicates. I found this very helpful - http://gurmitteotia.blogspot.co.uk/2015/06/specification-pattern-with-repository.html#more. His blog includes extension methods to chain predicate expressions and the pattern in general, & it is based around EF.</p>

Body:
I'm trying to create a way to allow users of my app to create their own filter views of data. To do that I'm composing a list of Key-Operator-Value pairs which I then try, after multiple steps, to parse into an expression to use with a Where call. So far, I can get to a point where I have the expressions created, but I can't figure out how to merge them into a single expression.

Could someone point me in the right direction on how to do so? From reading similar posts online, I've settled on trying to do it using the LINQKit PredicateBuilder, but all I'm getting back is the initial starting predicate. Here's the code I have so far, it is by no means final, I'm just prototyping it in LINQPad:

void Main() {
    //  1. Create form post object
    var query = new QueryEdit {
        Filters = new List<Filter> {
            new Filter {
                Id = 1,
                Key = "Name",
                Operator = "=",
                Value = "New York"
            },
            new Filter {
                Id = 2,
                Key = "Name",
                Operator = "!=",
                Value = "Boston"
            },
            new Filter {
                Id = 3,
                Key = "Name",
                Operator = "=",
                Value = "Washington"
            }
        },
        FilterClause = "1 AND 3 OR 2 OR 4"
    };

    //  2. Compose filter groups
    var filterGroups = GetFilterGroups(query);

    //  3. Compose expression groups
    var expressionGroups = GetExpressionGroups<Region>(filterGroups);

    expressionGroups.Dump();

    //  4. Compose full expression
    var expression = GetExpression(expressionGroups);

    expression.Dump();

    //  5. Serialize expression to blob
    //  6. Deserialize blob to expression
    //  7. Execute expression against db
}

static IEnumerable<FilterGroup> GetFilterGroups(
    QueryEdit query) {
    //  The parentheses are meaningless, so they need to be removed.
    //  The groups are all conditions with ANDs, separated by ORs.

    return query.FilterClause.Replace(")", string.Empty).Replace("(", string.Empty).Split(new[] { "OR" }, StringSplitOptions.None).Select(
        fc => new FilterGroup {
            Filters = fc.Split(new[] { "AND" }, StringSplitOptions.None).Select(
                i => {
                    var id = Convert.ToByte(i);

                    return query.Filters.SingleOrDefault(
                        f => f.Id == id);
                }).Where(
                f => f != null).OrderBy(
                f => f.Id)
        }).Where(
        fg => fg.Filters.Any());
}

static IEnumerable<IEnumerable<Expression<Func<T, bool>>>> GetExpressionGroups<T>(
    IEnumerable<FilterGroup> filterGroups) {
    var type = typeof(T);
    var parameter = Expression.Parameter(type);

    return filterGroups.Select(
        fg => {
            return fg.Filters.Select(
                f => {
                    var left = Expression.Property(parameter, type.GetProperty(f.Key));
                    var right = Expression.Constant(f.Value);
                    var body = GetExpressionBody(f.Operator, left, right);

                    if (body == null) {
                        return null;
                    }

                    return Expression.Lambda<Func<T, bool>>(body, parameter);
                }).Where(
                e => e != null);
        });
}

static object GetExpression<T>(
    IEnumerable<IEnumerable<Expression<Func<T, bool>>>> expressionGroups) {
    var predicate = PredicateBuilder.True<T>();

    foreach (var expressionGroup in expressionGroups) {
        var expression = GetPredicateForGroup<T>(expressionGroup);

        predicate.Or(expression);
    }

    return predicate;
}

static Expression<Func<T, bool>> GetPredicateForGroup<T>(
    IEnumerable<Expression<Func<T, bool>>> expressionGroup) {
    var predicate = PredicateBuilder.True<T>();

    foreach (var expression in expressionGroup) {
        predicate.And(expression);
    }

    return predicate;
}

static Expression GetExpressionBody(
    string @operator,
    Expression left,
    Expression right) {
    switch (@operator) {
        case "=":
            return Expression.Equal(left, right);
        case "!=":
            return Expression.NotEqual(left, right);
        case ">":
            return Expression.GreaterThan(left, right);
        case ">=":
            return Expression.GreaterThanOrEqual(left, right);
        case "<":
            return Expression.LessThan(left, right);
        case "<=":
            return Expression.LessThanOrEqual(left, right);
        default:
            return null;
    }
}

sealed class QueryEdit {
    public IEnumerable<Filter> Filters { get; set; }
    public string FilterClause { get; set; }
    public int Id { get; set; }
    public string Name { get; set; }
}

sealed class Filter {
    public byte Id { get; set; }
    public string Key { get; set; }
    public string Operator { get; set; }
    public object Value { get; set; }
}

sealed class FilterGroup {
    public IEnumerable<Filter> Filters { get; set; }
}

I'm also open to other suggestions on how to accomplish this, or overall improvements. Thanks in advance!


问:
Globally replace in Postgres JSONB field
I need to globally replace a particular string that occurs multiple places in a nested JSON structure, thats stored as jsonb in a postgres table.  For example:

```<code>{
  "location": "tmp/config",
  "alternate_location": {
    "name": "config",
    "location": "tmp/config"
  }
}
</code>```

...should become:

```<code>{
  "location": "tmp/new_config",
  "alternate_location": {
    "name": "config",
    "location": "tmp/new_config"
  }
}
</code>```

I've tried:

```<code>UPDATE files SET meta_data = to_json(replace(data::TEXT, 'tmp/config', 'tmp/new_config'));
</code>```

Unfortunately this results in malformed JSON, with triple escaped quotes.

Any ideas how to do this?

答:
Use a simple cast to jsonb instead of to_json(), e.g.:

```<code>with files(meta_data) as (
values(
'{
  "location": "tmp/config",
  "alternate_location": {
    "name": "config",
    "location": "tmp/config"
  }
}'::jsonb)
)

select replace(meta_data::text, 'tmp/config', 'tmp/new_config')::jsonb
from files;

                                                replace
--------------------------------------------------------------------------------------------------------
 {"location": "tmp/new_config", "alternate_location": {"name": "config", "location": "tmp/new_config"}}
(1 row)
</code>```

Body:
I need to globally replace a particular string that occurs multiple places in a nested JSON structure, thats stored as jsonb in a postgres table.  For example:

{
  "location": "tmp/config",
  "alternate_location": {
    "name": "config",
    "location": "tmp/config"
  }
}

...should become:

{
  "location": "tmp/new_config",
  "alternate_location": {
    "name": "config",
    "location": "tmp/new_config"
  }
}

I've tried:

UPDATE files SET meta_data = to_json(replace(data::TEXT, 'tmp/config', 'tmp/new_config'));

Unfortunately this results in malformed JSON, with triple escaped quotes.

Any ideas how to do this?

Body:
Use a simple cast to jsonb instead of to_json(), e.g.:

with files(meta_data) as (
values(
'{
  "location": "tmp/config",
  "alternate_location": {
    "name": "config",
    "location": "tmp/config"
  }
}'::jsonb)
)

select replace(meta_data::text, 'tmp/config', 'tmp/new_config')::jsonb
from files;

                                                replace
--------------------------------------------------------------------------------------------------------
 {"location": "tmp/new_config", "alternate_location": {"name": "config", "location": "tmp/new_config"}}
(1 row)


问:
exception ToString(): debug versus release mode
regarding th stack trace depth in debug and release mode. I have come to the following conclusion (calling exception.ToString()):

at debug mode you get the complete stack trace with line numbers (several frames) at release mode instead complete stack trace you get only the throwing method in try block (single frame) with line numbers

-is that true ?
-is there any way to get the complete stack tracve on release mode ?

答:
You get line numbers in stack trace if you have the PDB file associated with your assembly. Even in Release builds, you can have the compiler to generate the PDB files and ship them alongside your assembly.  You can use csc /debug:pdbonly when you're compiling your application. In Visual Studio 2010, you can use Project Properties -> Build -> Advanced -> Debug Info = pdb-only to achieve the same thing.

Update (stack trace depth):

You get full stack trace in Release builds.  However, compilers may inline methods and optimize tail calls, making the stack trace different.


问:
IF statement evaluating as "True" when condition criteria not met

Hi everyone.

I am trying to create a calculator that determines what action to take depending on the values of multiple gasses. Each gas will have a value within a cell that can fall within one of four ranges . Each gas is given a "priority rank" the range its value lies within. Then the highest priority determines what action to be taken.

The Problem:

The first If statement of each conditional block is being evaluated as "True" even if the value in the cell is greater than the value specified in the condition. How can I get the conditional statements to properly evaluate the condition based off the value in the cell.

Sub DGOA_Action_500kVA()

```<code>Dim HydrogenValue As Variant
Dim MethaneValue As Variant
Dim EthaneVaule As Variant
Dim EthyleneValue As Variant
Dim AcetyleneValue As Variant
Dim Action As Integer

Dim HydrogenPriority As Integer
Dim MethanePriority As Integer
Dim EthanePriority As Integer
Dim EthylenePriority As Integer
Dim AcetylenePriority As Integer

Worksheets("500 kVa").Activate
HydrogenValue = Worksheets("500 kVa").Range("B3").Select
MethaneValue = Worksheets("500 kVa").Range("C3").Select
EthaneVaule = Worksheets("500 kVa").Range("F3").Select
EthyleneValue = Worksheets("500 kVa").Range("G3").Select
AcetyleneValue = Worksheets("500 kVa").Range("H3").Select

Dim PriorityRange As Range
Set PriorityRange = Worksheets("500 kVa").Range("J3:P3")

If HydrogenValue < 2000 Then
    HydrogenPriority = 1
    MsgBox (Worksheets("500 kVa").Range("B6").Select)

    ElseIf HydrogenValue >= 2000 And HydrogenValue < 20000 Then
        HydrogenPriority = 2

    ElseIf HydrogenValue >= 20000 And HydrogenValue < 27000 Then
        HydrogenPriority = 3

    ElseIf HydrogenValue >= 27000 Then
        HydrogenPriority = 4
End If

If MethaneValue < 2000 Then
    MethanePriority = 1

    ElseIf MethaneValue >= 2000 And MethaneValue < 15000 Then
        MethanePriority = 2

    ElseIf MethaneValue >= 15000 Then
        MethanePriority = 4
End If

If EthaneVaule < 1000 Then
    EthanePriority = 1

    ElseIf EthaneVaule >= 1000 And EthaneVaule < 2250 Then
        EthanePriority = 3

    ElseIf EthaneVaule <= 2250 Then
        EthanePriority = 4
End If

If EthyleneValue < 1000 Then
    EthylenePriority = 1

    ElseIf EthyleneValue >= 1000 And EthyleneValue < 3000 Then
        EthylenePriority = 3

    ElseIf EthyleneValue >= 2250 Then
        EthylenePriority = 4

    Else
        EthylenePriority = 1
End If

If AcetyleneValue < 1 Then
    AcetylenePriority = 1

    ElseIf AcetyleneValue >= 1 And AcetyleneValue < 20 Then
        AcetylenePriority = 2

    ElseIf AcetyleneValue >= 20 And AcetyleneValue < 100 Then
        AcetylenePriority = 3

    ElseIf AcetyleneValue >= 100 Then
        AcetylenePriority = 4

    Else
        AcetylenePriority = 1

End If

Worksheets("500 kVa").Range("J3").Value = HydrogenPriority
Worksheets("500 kVa").Range("K3").Value = MethanePriority
Worksheets("500 kVa").Range("N3").Value = EthanePriority
Worksheets("500 kVa").Range("O3").Value = EthylenePriority
Worksheets("500 kVa").Range("P3").Value = AcetylenePriority

Action = Application.WorksheetFunction.Max(PriorityRange)

Select Case Action

    Case 1
        Worksheets("Sheet1").Range("AX4").Value = "Normal"
    Case 2
        Worksheets("Sheet1").Range("AX4").Value = "Add in Watch List"
    Case 3
        Worksheets("Sheet1").Range("AX4").Value = "Resample"
    Case 4
        Worksheets("Sheet1").Range("AX4").Value = "Engineering Evaluation"
    Case Else
        Worksheets("Sheet1").Range("AX4").Value = "Error"
End Select

Worksheets(1).Activate
MsgBox ("The Action Has Been Determined")
</code>```

End Sub

答:
I realized that I was using "Select" property by accident instead of the "Value" property to get the contents within the cell.

Problem solved

Body:
Hi everyone.

I am trying to create a calculator that determines what action to take depending on the values of multiple gasses. Each gas will have a value within a cell that can fall within one of four ranges . Each gas is given a "priority rank" the range its value lies within. Then the highest priority determines what action to be taken.

The Problem:

The first If statement of each conditional block is being evaluated as "True" even if the value in the cell is greater than the value specified in the condition. How can I get the conditional statements to properly evaluate the condition based off the value in the cell.

Sub DGOA_Action_500kVA()

Dim HydrogenValue As Variant
Dim MethaneValue As Variant
Dim EthaneVaule As Variant
Dim EthyleneValue As Variant
Dim AcetyleneValue As Variant
Dim Action As Integer

Dim HydrogenPriority As Integer
Dim MethanePriority As Integer
Dim EthanePriority As Integer
Dim EthylenePriority As Integer
Dim AcetylenePriority As Integer

Worksheets("500 kVa").Activate
HydrogenValue = Worksheets("500 kVa").Range("B3").Select
MethaneValue = Worksheets("500 kVa").Range("C3").Select
EthaneVaule = Worksheets("500 kVa").Range("F3").Select
EthyleneValue = Worksheets("500 kVa").Range("G3").Select
AcetyleneValue = Worksheets("500 kVa").Range("H3").Select

Dim PriorityRange As Range
Set PriorityRange = Worksheets("500 kVa").Range("J3:P3")

If HydrogenValue < 2000 Then
    HydrogenPriority = 1
    MsgBox (Worksheets("500 kVa").Range("B6").Select)

    ElseIf HydrogenValue >= 2000 And HydrogenValue < 20000 Then
        HydrogenPriority = 2

    ElseIf HydrogenValue >= 20000 And HydrogenValue < 27000 Then
        HydrogenPriority = 3

    ElseIf HydrogenValue >= 27000 Then
        HydrogenPriority = 4
End If

If MethaneValue < 2000 Then
    MethanePriority = 1

    ElseIf MethaneValue >= 2000 And MethaneValue < 15000 Then
        MethanePriority = 2

    ElseIf MethaneValue >= 15000 Then
        MethanePriority = 4
End If

If EthaneVaule < 1000 Then
    EthanePriority = 1

    ElseIf EthaneVaule >= 1000 And EthaneVaule < 2250 Then
        EthanePriority = 3

    ElseIf EthaneVaule <= 2250 Then
        EthanePriority = 4
End If

If EthyleneValue < 1000 Then
    EthylenePriority = 1

    ElseIf EthyleneValue >= 1000 And EthyleneValue < 3000 Then
        EthylenePriority = 3

    ElseIf EthyleneValue >= 2250 Then
        EthylenePriority = 4

    Else
        EthylenePriority = 1
End If

If AcetyleneValue < 1 Then
    AcetylenePriority = 1

    ElseIf AcetyleneValue >= 1 And AcetyleneValue < 20 Then
        AcetylenePriority = 2

    ElseIf AcetyleneValue >= 20 And AcetyleneValue < 100 Then
        AcetylenePriority = 3

    ElseIf AcetyleneValue >= 100 Then
        AcetylenePriority = 4

    Else
        AcetylenePriority = 1

End If

Worksheets("500 kVa").Range("J3").Value = HydrogenPriority
Worksheets("500 kVa").Range("K3").Value = MethanePriority
Worksheets("500 kVa").Range("N3").Value = EthanePriority
Worksheets("500 kVa").Range("O3").Value = EthylenePriority
Worksheets("500 kVa").Range("P3").Value = AcetylenePriority

Action = Application.WorksheetFunction.Max(PriorityRange)

Select Case Action

    Case 1
        Worksheets("Sheet1").Range("AX4").Value = "Normal"
    Case 2
        Worksheets("Sheet1").Range("AX4").Value = "Add in Watch List"
    Case 3
        Worksheets("Sheet1").Range("AX4").Value = "Resample"
    Case 4
        Worksheets("Sheet1").Range("AX4").Value = "Engineering Evaluation"
    Case Else
        Worksheets("Sheet1").Range("AX4").Value = "Error"
End Select

Worksheets(1).Activate
MsgBox ("The Action Has Been Determined")

End Sub


问:
How can I properly use a Pandas Dataframe with a multiindex that includes Intervals?
I'm trying to slice into a DataFrame that has a MultiIndex composed of an IntervalIndex and a regular Index. Example code:

```<code>from pandas import Interval as ntv

df = pd.DataFrame.from_records([
   {'id': 1, 'var1': 0.1, 'ntv': ntv(0,10), 'E': 1},
   {'id':2, 'var1': 0.5, 'ntv': ntv(0,12), 'E': 0}
], index=('ntv', 'id'))
</code>```

Looks like this:

```<code>            E  var1
ntv     id
(0, 10] 1   1   0.1
(0, 12] 2   0   0.5
</code>```

What I would like to do is to slice into the DataFrame at a specific value and return all rows that has an interval that contains the value. Ex:

```<code>df.loc[4]
</code>```

should return (trivially)

```<code>    E  var1
id
1   1   0.1
2   0   0.5
</code>```

The problem is I keep getting a TypeError about the index, and the docs show a similar operation (but on a single-level index) that does produce what I'm looking for.

```<code>TypeError: only integer scalar arrays can be converted to a scalar index
</code>```

I've tried many things, nothing seems to work normally. I could include the id column inside the dataframe, but I'd rather keep my index unique, and I would constantly be calling set_index('id').

I feel like either a) I'm missing something about MultiIndexes or b) there is a bug / ambiguity with using an IntervalIndex in a MultiIndex.

答:
```<code>def check_value(num):
    return df[[num in i for i in map(lambda x: x[0], df.index)]]

a = check_value(4)
a
>>
            E  var1
ntv     id
(0, 10] 1   1   0.1
(0, 12] 2   0   0.5
</code>```

if you want to drop the index level, you can add

```<code>a.index = a.droplevel(0)
</code>```

Body:
I'm trying to slice into a DataFrame that has a MultiIndex composed of an IntervalIndex and a regular Index. Example code:

from pandas import Interval as ntv

df = pd.DataFrame.from_records([
   {'id': 1, 'var1': 0.1, 'ntv': ntv(0,10), 'E': 1},
   {'id':2, 'var1': 0.5, 'ntv': ntv(0,12), 'E': 0}
], index=('ntv', 'id'))

Looks like this:

            E  var1
ntv     id
(0, 10] 1   1   0.1
(0, 12] 2   0   0.5

What I would like to do is to slice into the DataFrame at a specific value and return all rows that has an interval that contains the value. Ex:

df.loc[4]

should return (trivially)

    E  var1
id
1   1   0.1
2   0   0.5

The problem is I keep getting a TypeError about the index, and the docs show a similar operation (but on a single-level index) that does produce what I'm looking for.

TypeError: only integer scalar arrays can be converted to a scalar index

I've tried many things, nothing seems to work normally. I could include the id column inside the dataframe, but I'd rather keep my index unique, and I would constantly be calling set_index('id').

I feel like either a) I'm missing something about MultiIndexes or b) there is a bug / ambiguity with using an IntervalIndex in a MultiIndex.

Body:
def check_value(num):
    return df[[num in i for i in map(lambda x: x[0], df.index)]]

a = check_value(4)
a
>>
            E  var1
ntv     id
(0, 10] 1   1   0.1
(0, 12] 2   0   0.5

if you want to drop the index level, you can add

a.index = a.droplevel(0)


问:
TFS, is there a variable for the artifact zip file name?
Is there a way to get the artifact file name easily? Looks like TFS already knows this as shown in the log below:</p>

```<code>No path specified for search pattern: *.zip defaulting to: F:\TFS2017_Release_Agent\_work\r1\a

2017-12-08T16:38:36.8519067Z Searching for: *.zip under directory: F:\TFS2017_Release_Agent\_work\r1\a

2017-12-08T16:38:36.8519067Z Found: 1 files to extract:

2017-12-08T16:38:36.8519067Z F:\TFS2017_Release_Agent\_work\r1\a\ZFJ0_ServiceSearchPlusBuildDefn\BETALink.Service.SearchPlus\ZFJ0_BETALink10.4 DevOps_BETALink.Service.SearchPlus_1.0.0_886.zip

2017-12-08T16:38:36.8519067Z Creating destination folder: F:\TFS2017_Release_Agent\_work\r1\a\48

2017-12-08T16:38:36.8519067Z Extracting file: F:\TFS2017_Release_Agent\_work\r1\a\ZFJ0_ServiceSearchPlusBuildDefn\BETALink.Service.SearchPlus\ZFJ0_BETALink10.4 DevOps_BETALink.Service.SearchPlus_1.0.0_886.zip

2017-12-08T16:38:36.8519067Z [command]F:\TFS2017_Release_Agent\_work\_tasks\ExtractFiles_5e1e3830-fbfb-11e5-aab1-090c92bc4988\1.112.1\7zip\7z.exe x -oF:\TFS2017_Release_Agent\_work\r1\a\48 F:\TFS2017_Release_Agent\_work\r1\a\ZFJ0_ServiceSearchPlusBuildDefn\BETALink.Service.SearchPlus\ZFJ0_BETALink10.4 DevOps_BETALink.Service.SearchPlus_1.0.0_886.zip

2017-12-08T16:38:36.8987827Z
</code>```

答:
For others, I have to write a couple of lines to get the zip file name. ZFJ0_ServiceSearchPlusBuildDefn.DefinitionName is my {Artifact alias}:</p>

```<code>$zipdir = "$(System.DefaultWorkingDirectory)\$(Release.Artifacts.ZFJ0_ServiceSearchPlusBuildDefn.DefinitionName)\$(Release.DefinitionName)"
Write-Host "zipfile directory: "$zipdir

$zipfile = Get-ChildItem -Filter *.zip "$zipdir"
Write-Host "zipfile : "$zipfile
</code>```

Body:
Is there a way to get the artifact file name easily? Looks like TFS already knows this as shown in the log below:

No path specified for search pattern: *.zip defaulting to: F:\TFS2017_Release_Agent\_work\r1\a

2017-12-08T16:38:36.8519067Z Searching for: *.zip under directory: F:\TFS2017_Release_Agent\_work\r1\a

2017-12-08T16:38:36.8519067Z Found: 1 files to extract:

2017-12-08T16:38:36.8519067Z F:\TFS2017_Release_Agent\_work\r1\a\ZFJ0_ServiceSearchPlusBuildDefn\BETALink.Service.SearchPlus\ZFJ0_BETALink10.4 DevOps_BETALink.Service.SearchPlus_1.0.0_886.zip

2017-12-08T16:38:36.8519067Z Creating destination folder: F:\TFS2017_Release_Agent\_work\r1\a\48

2017-12-08T16:38:36.8519067Z Extracting file: F:\TFS2017_Release_Agent\_work\r1\a\ZFJ0_ServiceSearchPlusBuildDefn\BETALink.Service.SearchPlus\ZFJ0_BETALink10.4 DevOps_BETALink.Service.SearchPlus_1.0.0_886.zip

2017-12-08T16:38:36.8519067Z [command]F:\TFS2017_Release_Agent\_work\_tasks\ExtractFiles_5e1e3830-fbfb-11e5-aab1-090c92bc4988\1.112.1\7zip\7z.exe x -oF:\TFS2017_Release_Agent\_work\r1\a\48 F:\TFS2017_Release_Agent\_work\r1\a\ZFJ0_ServiceSearchPlusBuildDefn\BETALink.Service.SearchPlus\ZFJ0_BETALink10.4 DevOps_BETALink.Service.SearchPlus_1.0.0_886.zip

2017-12-08T16:38:36.8987827Z

Body:
For others, I have to write a couple of lines to get the zip file name. ZFJ0_ServiceSearchPlusBuildDefn.DefinitionName is my {Artifact alias}:

$zipdir = "$(System.DefaultWorkingDirectory)\$(Release.Artifacts.ZFJ0_ServiceSearchPlusBuildDefn.DefinitionName)\$(Release.DefinitionName)"
Write-Host "zipfile directory: "$zipdir

$zipfile = Get-ChildItem -Filter *.zip "$zipdir"
Write-Host "zipfile : "$zipfile


问:
Javascript passing var on click in a table
Trying to pass a variable when a
 in clicked, am getting "undefined".
Works if I remove the onclick="fnselect()" and use a submit button, which I don't want.

function highlight(e) {
  if (selected[0]) selected[0].className = '';
  e.target.parentNode.className = 'selected';
}

var table = document.getElementById('table'),
  selected = table.getElementsByClassName('selected');
table.onclick = highlight;

function fnselect(){
  var phpValue = $("tr.selected td:first" ).html();
  window.location.href = "http://website/test.php?phpValue="+phpValue;
}```
<pre class="snippet-code-html lang-html prettyprint-override"><code><table id="table">
     <tr onclick="fnselect()">
      <td>1 Ferrari F138</td>
      <td>1 000€</td>
      <td>1 200€</td>
      <td>Model monopostu stajne Scuderia Ferrari pre sezónu 2013</td>
      <td>1</td>
      <td>F138</td>
      <td>Klik pre detaily</td>
     </tr>
     <tr onclick="fnselect()">
      <td>2 Ferrari F138</td>
      <td>1 000€</td>
      <td>1 200€</td>
      <td>Model monopostu stajne Scuderia Ferrari pre sezónu 2013</td>
      <td>1</td>
      <td>F138</td>
      <td>Klik pre detaily</td>
     </tr>
     <tr onclick="fnselect()">
      <td>3 Ferrari F138</td>
      <td>1 000€</td>
      <td>1 200€</td>
      <td>Model monopostu stajne Scuderia Ferrari pre sezónu 2013</td>
      <td>1</td>
      <td>F138</td>
      <td>Klik pre detaily</td>
     </tr>
    </table></code>```

答:
tr does not become selected when you click it. You can pass the tr object to function instead:

```<code><tr onclick="fnselect(this)">
</code>```

and

```<code>function fnselect(obj){
  var phpValue = obj.cells[0].innerHTML;
  window.location.href = "http://website/test.php?phpValue="+phpValue;
}
</code>```

Body:
Trying to pass a variable when a
 in clicked, am getting "undefined".
Works if I remove the onclick="fnselect()" and use a submit button, which I don't want.

function highlight(e) {
  if (selected[0]) selected[0].className = '';
  e.target.parentNode.className = 'selected';
}

var table = document.getElementById('table'),
  selected = table.getElementsByClassName('selected');
table.onclick = highlight;

function fnselect(){
  var phpValue = $("tr.selected td:first" ).html();
  window.location.href = "http://website/test.php?phpValue="+phpValue;
}

      1 Ferrari F138
      1 000€
      1 200€
      Model monopostu stajne Scuderia Ferrari pre sezónu 2013
      1
      F138
      Klik pre detaily

      2 Ferrari F138
      1 000€
      1 200€
      Model monopostu stajne Scuderia Ferrari pre sezónu 2013
      1
      F138
      Klik pre detaily

      3 Ferrari F138
      1 000€
      1 200€
      Model monopostu stajne Scuderia Ferrari pre sezónu 2013
      1
      F138
      Klik pre detaily

Body:
tr does not become selected when you click it. You can pass the tr object to function instead:

<tr onclick="fnselect(this)">

and

function fnselect(obj){
  var phpValue = obj.cells[0].innerHTML;
  window.location.href = "http://website/test.php?phpValue="+phpValue;
}


问:
How do I clear service worker caches when unregistering?
Popular question How do I uninstall a Service Worker? has a fine answer for unregistering a service worker using JavaScript. How can I get the service worker to clear any named caches when it unregisters?

From MDN unregister, The service worker will finish any ongoing operations before it is unregistered. So, it sounds like it should be possible to listen for an uninstall event, but no such event exists as far as I can tell. For giggles, I tried adding self.addEventListener('uninstall', ...) to the service worker, but as expected, it did not fire on unregister. I also tried listening for any message that might be sent on unregister... no dice.

I also considered watching the service worker state, but none of installing, installed, activating, activated, and redundant appear relevant. Not to mention, it doesn't look like message passing to the service worker is possible, so I'm not sure what I'd do with an uninstalling state from outside the service worker anyways.

Reaching for straws, while I see references to an uninstalling flag, I haven't seen it exposed anywhere. Suggestions?

答:
This can be accomplished with message passing. One reason I prefer this to the answer by Jeff Posnick is that the name of the cache does not need to be known outside of the service worker. In the service worker, add a message listener:

```<code>self.addEventListener('message', (event) => {
    messageHandler(event);
});
</code>```

The messageHandler() can then take action on various passed messages. For example:

```<code>function messageHandler(event) {
    if (event.data === 'purge_cache') {
        caches.delete('mycache')
        .then((success) => {
            console.log("Cache removal status: " + success);
        })
        .catch((err) => {
            console.log("Cache removal error: ", err);
        });
    }
}
</code>```

To trigger this action, post the message from your script on a page that is under the service worker's scope. So, for the original question of how to delete the cache and then unregister the service worker, you could do this:

```<code>navigator.serviceWorker.getRegistration()
.then((registration) => {
    registration.active.postMessage('purge_cache');
    registration.unregister()
    .then((success) => {
        console.log('Service worker unregistration status: ' + success);
    })
    .catch((err) => {
        console.log('Service worker unregistration failed', err);
    });
})
.catch((err) => {
    console.log('Service worker registration not found.');
});
</code>```

Notice, I did not wait for a successful reply from postMessage('purge_cache'); from MDN unregister, "The service worker will finish any ongoing operations before it is unregistered."

Body:
This can be accomplished with message passing. One reason I prefer this to the answer by Jeff Posnick is that the name of the cache does not need to be known outside of the service worker. In the service worker, add a message listener:

self.addEventListener('message', (event) => {
    messageHandler(event);
});

The messageHandler() can then take action on various passed messages. For example:

function messageHandler(event) {
    if (event.data === 'purge_cache') {
        caches.delete('mycache')
        .then((success) => {
            console.log("Cache removal status: " + success);
        })
        .catch((err) => {
            console.log("Cache removal error: ", err);
        });
    }
}

To trigger this action, post the message from your script on a page that is under the service worker's scope. So, for the original question of how to delete the cache and then unregister the service worker, you could do this:

navigator.serviceWorker.getRegistration()
.then((registration) => {
    registration.active.postMessage('purge_cache');
    registration.unregister()
    .then((success) => {
        console.log('Service worker unregistration status: ' + success);
    })
    .catch((err) => {
        console.log('Service worker unregistration failed', err);
    });
})
.catch((err) => {
    console.log('Service worker registration not found.');
});

Notice, I did not wait for a successful reply from postMessage('purge_cache'); from MDN unregister, "The service worker will finish any ongoing operations before it is unregistered."


问:
Encrypting in Node JS using AES 256 CTR and Decrypting in Golang
I have encrypted some JSON Text using following Node JS code:-

```<code>var algorithm = 'aes-256-ctr';
var crypto = require('crypto');

var password = "6A80FD8D38D579D1090F6CDB62C729311781E4BA31CD7D804BD7BF5AEC3BFC2D"

var typedRequest = {"abc":"cde"}

var cipher = crypto.createCipher(algorithm, password);
var hashRequest = cipher.update(JSON.stringify(typedRequest),
    'utf8', 'hex');
hashRequest += cipher.final('hex');
</code>```

And now, I wanted to decrypt this encryptedText in Golang. But I am not able to found any way to do this, since in almost all decryption logic examples of AES 256 CTR in Golang, I found it always requires IV while decrypting but I have not use the same in Node JS. I have written something in Golang but its not decrypting correctly and giving error for now:-

```<code>package main

import (
    "crypto/aes"
    "crypto/cipher"
    "encoding/hex"
    "fmt"

)

func main() {

    encKey := "6A80FD8D38D579D1090F6CDB62C729311781E4BA31CD7D804BD7BF5AEC3BFC2D"
    cipherText := "746c17cd10f8f86646f843ac2a"

    encKeyDecoded, err := hex.DecodeString(encKey)
    if err != nil {
        panic(err)
    }
    cipherTextDecoded, err := hex.DecodeString(cipherText)
    if err != nil {
        panic(err)
    }

    iv := cipherTextDecoded[:aes.BlockSize]

    block, err := aes.NewCipher([]byte(encKeyDecoded))
    if err != nil {
        panic(err)
    }

    cipherTextBytes := []byte(cipherTextDecoded)

    plaintext := make([]byte, len(cipherTextBytes) - aes.BlockSize)
    stream := cipher.NewCTR(block, iv)
    stream.XORKeyStream(plaintext, cipherTextBytes[aes.BlockSize:])

    fmt.Println(string(plaintext))
}
</code>```

Any help in getting the correct Golang code is highly appreciated. Thanks

=====================================

Now, I have updated to following after taking suggestion from Answer:-

This is my node js code:-

```<code>var crypto = require('crypto'),
  algorithm = 'aes-256-ctr',
  password = '6A80FD8D38D579D1090F6CDB62CA34CA',
  // do not use a global iv for production,
  // generate a new one for each encryption
  iv = '79b67e539e7fcadf'

var typedRequest = {"abc":"cde"}

var cipher = crypto.createCipheriv(algorithm, password, iv);
var hashRequest = cipher.update(JSON.stringify(typedRequest),
    'utf8', 'hex');
hashRequest += iv.toString('hex') + cipher.final('hex');
</code>```

This is my Go code:-

```<code>package main

import (
    "crypto/aes"
    "crypto/cipher"
    "encoding/hex"
    "fmt"

)

func main() {

    encKey := "6A80FD8D38D579D1090F6CDB62CA34CA"
    cipherText := "af7d1eb42107549a7e3adbce1a79b67e539e7fcadf" // Got from above

    encKeyDecoded, err := hex.DecodeString(encKey)
    if err != nil {
        panic(err)
    }
    cipherTextDecoded, err := hex.DecodeString(cipherText)
    if err != nil {
        panic(err)
    }

    block, err := aes.NewCipher([]byte(encKeyDecoded))
    if err != nil {
        panic(err)
    }

    iv := cipherTextDecoded[:aes.BlockSize]
    cipherTextBytes := []byte(cipherTextDecoded)

    plaintext := make([]byte, len(cipherTextBytes) - aes.BlockSize)
    stream := cipher.NewCTR(block, iv)
    stream.XORKeyStream(plaintext, cipherTextBytes[aes.BlockSize:])

    fmt.Println(string(plaintext))
}
</code>```

Now, I am getting something completely different in decrypted form.

答:
It is your NodeJS code that is incorrect.  CTR mode requires an IV, calling crypto.createCipher is undefined behavior for CTR mode.

As per the NodeJS docs for Crypto, you should be using crypto.createCipheriv.  Your Golang code attempts to retrieve this IV from the start of the plaintext, so you'll need to place it there in your NodeJS code.

The IV should be unique for each encryption operation, using a CSPRNG is the recommended way to do this.

Body:
I have encrypted some JSON Text using following Node JS code:-

var algorithm = 'aes-256-ctr';
var crypto = require('crypto');

var password = "6A80FD8D38D579D1090F6CDB62C729311781E4BA31CD7D804BD7BF5AEC3BFC2D"

var typedRequest = {"abc":"cde"}

var cipher = crypto.createCipher(algorithm, password);
var hashRequest = cipher.update(JSON.stringify(typedRequest),
    'utf8', 'hex');
hashRequest += cipher.final('hex');

And now, I wanted to decrypt this encryptedText in Golang. But I am not able to found any way to do this, since in almost all decryption logic examples of AES 256 CTR in Golang, I found it always requires IV while decrypting but I have not use the same in Node JS. I have written something in Golang but its not decrypting correctly and giving error for now:-

package main

import (
    "crypto/aes"
    "crypto/cipher"
    "encoding/hex"
    "fmt"

)

func main() {

    encKey := "6A80FD8D38D579D1090F6CDB62C729311781E4BA31CD7D804BD7BF5AEC3BFC2D"
    cipherText := "746c17cd10f8f86646f843ac2a"

    encKeyDecoded, err := hex.DecodeString(encKey)
    if err != nil {
        panic(err)
    }
    cipherTextDecoded, err := hex.DecodeString(cipherText)
    if err != nil {
        panic(err)
    }

    iv := cipherTextDecoded[:aes.BlockSize]

    block, err := aes.NewCipher([]byte(encKeyDecoded))
    if err != nil {
        panic(err)
    }

    cipherTextBytes := []byte(cipherTextDecoded)

    plaintext := make([]byte, len(cipherTextBytes) - aes.BlockSize)
    stream := cipher.NewCTR(block, iv)
    stream.XORKeyStream(plaintext, cipherTextBytes[aes.BlockSize:])

    fmt.Println(string(plaintext))
}

Any help in getting the correct Golang code is highly appreciated. Thanks

=====================================

Now, I have updated to following after taking suggestion from Answer:-

This is my node js code:-

var crypto = require('crypto'),
  algorithm = 'aes-256-ctr',
  password = '6A80FD8D38D579D1090F6CDB62CA34CA',
  // do not use a global iv for production,
  // generate a new one for each encryption
  iv = '79b67e539e7fcadf'

var typedRequest = {"abc":"cde"}

var cipher = crypto.createCipheriv(algorithm, password, iv);
var hashRequest = cipher.update(JSON.stringify(typedRequest),
    'utf8', 'hex');
hashRequest += iv.toString('hex') + cipher.final('hex');

This is my Go code:-

package main

import (
    "crypto/aes"
    "crypto/cipher"
    "encoding/hex"
    "fmt"

)

func main() {

    encKey := "6A80FD8D38D579D1090F6CDB62CA34CA"
    cipherText := "af7d1eb42107549a7e3adbce1a79b67e539e7fcadf" // Got from above

    encKeyDecoded, err := hex.DecodeString(encKey)
    if err != nil {
        panic(err)
    }
    cipherTextDecoded, err := hex.DecodeString(cipherText)
    if err != nil {
        panic(err)
    }

    block, err := aes.NewCipher([]byte(encKeyDecoded))
    if err != nil {
        panic(err)
    }

    iv := cipherTextDecoded[:aes.BlockSize]
    cipherTextBytes := []byte(cipherTextDecoded)

    plaintext := make([]byte, len(cipherTextBytes) - aes.BlockSize)
    stream := cipher.NewCTR(block, iv)
    stream.XORKeyStream(plaintext, cipherTextBytes[aes.BlockSize:])

    fmt.Println(string(plaintext))
}

Now, I am getting something completely different in decrypted form.


问:
How to search in nested JSON data in vuejs?
I have nested JSON data like:

```<code>[{"id":"1","province_id":"ABC","name":"City One"},{"id":"2","province_id":"EFG","name":"City Two"}]
</code>```

I want to filter the JSON by province_id and put it in another variable. Is there any solutions in VueJS such as Vue.filter(); ?
I know we have "linq" which does the job but I do not want to use it.

答:
I think this will work for you:

var nestedJson = `[{"id":"1","province_id":"ABC","name":"City One"},{"id":"2","province_id":"EFG","name":"City Two"}]`;
var array = JSON.parse(nestedJson);

array = array.map(e => e["province_id"]);

console.log(array);```

Body:
I have nested JSON data like:

[{"id":"1","province_id":"ABC","name":"City One"},{"id":"2","province_id":"EFG","name":"City Two"}]

I want to filter the JSON by province_id and put it in another variable. Is there any solutions in VueJS such as Vue.filter(); ?
I know we have "linq" which does the job but I do not want to use it.


问:
where is Visual Studio.Uml.extension.dll for code generation in visual studio 2010?
when trying text template there is a line
import Visual Studio.Uml.extension.dll

but reference add can not find this dll. where is it?

答:
This was changed in RC to Microsoft.VisualStudio.ArchitectureTools.Extensibility.dll. The docs for it are here:

http://msdn.microsoft.com/en-us/library/ee517354%28VS.100%29.aspx

Hope that helps


问:
Android Studio Gradle Sync Error using google play-services-maps 11.6.2
I am developing a google map app using Android Studio.  When I sync the project with Gradle Files (Select Tools/Android/Sync Project with Gradle Files on Android Studio), I get the following error:

Could not resolve com.google.android.gms:play-services-maps:11.6.2.
Couold not get resource 'https://dl.google.com/dl/android/maven2/com/google/android/gms/play-services-maps/11.62./play-services-maps-11.6.2.pom

This is the software I am using:

Android Studio 3.0.1

Android Version 7.1.1

Google Play Services 46

Google Repository 58 (under Support Repository)

I've tried using both the google maps project template and the empty activity project but I get the same error.

I am using the http proxy setup I used from earlier projects.  I am able to install the play-services-maps and the google repository.

I also added the maven url to the Gradle Project file in the All Projects/Repositories section.

I have tried the suggestions from Stack Overflow but none have not worked.

Any ideas?

```<code>Gradle Project Build file:

// Top-level build file where you can add configuration options common to all sub-projects/modules.

buildscript {

    repositories {
        google()
        jcenter()
    }
    dependencies {
        classpath 'com.android.tools.build:gradle:3.0.1'

        // NOTE: Do not place your application dependencies here; they belong
        // in the individual module build.gradle files
    }
}

allprojects {
    repositories {
        google()
        jcenter()
        maven {
            url "https://maven.google.com"
        }
    }
}

task clean(type: Delete) {
    delete rootProject.buildDir
}
</code>```

The gradle app build file:

```<code>apply plugin: 'com.android.application'

android {
    compileSdkVersion 26
    defaultConfig {
        applicationId "hansen.scott.googlemaps"
        minSdkVersion 25
        targetSdkVersion 26
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
}

dependencies {
    implementation fileTree(dir: 'libs', include: ['*.jar'])
    implementation 'com.android.support:appcompat-v7:26.1.0'
    implementation 'com.android.support.constraint:constraint-layout:1.0.2'
    implementation 'com.google.android.gms:play-services-maps:11.6.2'
    testImplementation 'junit:junit:4.12'
    androidTestImplementation 'com.android.support.test:runner:1.0.1'
    androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.1'
}
</code>```

The error is in the line:
implementation 'com.google.android.gms:play-services-maps:11.6.2'

答:
Someone had the same issue I did and entered it on Stack Overflow a day or two ago.  The user was getting a build error when using player-services-maps 11.6.2.  If the user changed the version to 11.0.0, the build succeeded.

Once I back-dated the maps version to 11.0.0, this solved my problem too.  I created a new Google Map project and didn't modify the Gradle Build files.

The problem appears to be that play-services-maps won't build using a versions after 11.0.0.  I will post the issue on Github.

Thank you for everybody's help.

Body:
I am developing a google map app using Android Studio.  When I sync the project with Gradle Files (Select Tools/Android/Sync Project with Gradle Files on Android Studio), I get the following error:

Could not resolve com.google.android.gms:play-services-maps:11.6.2.
Couold not get resource 'https://dl.google.com/dl/android/maven2/com/google/android/gms/play-services-maps/11.62./play-services-maps-11.6.2.pom

This is the software I am using:

Android Studio 3.0.1

Android Version 7.1.1

Google Play Services 46

Google Repository 58 (under Support Repository)

I've tried using both the google maps project template and the empty activity project but I get the same error.

I am using the http proxy setup I used from earlier projects.  I am able to install the play-services-maps and the google repository.

I also added the maven url to the Gradle Project file in the All Projects/Repositories section.

I have tried the suggestions from Stack Overflow but none have not worked.

Any ideas?

Gradle Project Build file:

// Top-level build file where you can add configuration options common to all sub-projects/modules.

buildscript {

    repositories {
        google()
        jcenter()
    }
    dependencies {
        classpath 'com.android.tools.build:gradle:3.0.1'

        // NOTE: Do not place your application dependencies here; they belong
        // in the individual module build.gradle files
    }
}

allprojects {
    repositories {
        google()
        jcenter()
        maven {
            url "https://maven.google.com"
        }
    }
}

task clean(type: Delete) {
    delete rootProject.buildDir
}

The gradle app build file:

apply plugin: 'com.android.application'

android {
    compileSdkVersion 26
    defaultConfig {
        applicationId "hansen.scott.googlemaps"
        minSdkVersion 25
        targetSdkVersion 26
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
}

dependencies {
    implementation fileTree(dir: 'libs', include: ['*.jar'])
    implementation 'com.android.support:appcompat-v7:26.1.0'
    implementation 'com.android.support.constraint:constraint-layout:1.0.2'
    implementation 'com.google.android.gms:play-services-maps:11.6.2'
    testImplementation 'junit:junit:4.12'
    androidTestImplementation 'com.android.support.test:runner:1.0.1'
    androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.1'
}

The error is in the line:
implementation 'com.google.android.gms:play-services-maps:11.6.2'


问:
Why postfixed has higher precedence than prefixed?
Case 1:

```<code>int i = 10;
int j = i++;
</code>```

Here, first the value of i is assigned to j and than i is increased by 1.

Case 2:

```<code>int i = 10;
int j = ++i;
</code>```

Here, first i is increased by 1 and than it is assigned to j.

So, If the operation of increment is done first in the prefixed form, then why postfixed has higher precedence than prefixed?

答:
Linux manual page of operator define as same priority for both pre/post increment.

```<code>! ~ ++ -- + - (type) * & sizeof      right to left
</code>```

post increment rule is first assigned and then increment

```<code>int j = i++;// here first whatever i value is there that will be assigned to j
</code>```

Pre increment rule is first increment and then assign

```<code>int j = ++i;//++i itself will change i value & then modfied value will assign to j.
</code>```

for e.g consider below example

```<code>#include<stdio.h>
int main()
{
        int x = 10;
        int *p = &x;// assume p holds 0x100
        printf("x = %d *p = %d p = %p\n",x,*p,p);
        ++*p++;
        /** in above statement ++ and * having same precedence,then you should check assocaitivity which is R to L
          So start solving from Right to Left, whichever operator came first , solve that one first
          first *p++ came , again solve p++ first, which is post increment,so address will be same
         *0x100  means = 10
         now ++10 means = 11
         **/
        printf("x = %d *p = %d p = %p\n",x,*p,p);
}
</code>```

Body:
Case 1:

int i = 10;
int j = i++;

Here, first the value of i is assigned to j and than i is increased by 1.

Case 2:

int i = 10;
int j = ++i;

Here, first i is increased by 1 and than it is assigned to j.

So, If the operation of increment is done first in the prefixed form, then why postfixed has higher precedence than prefixed?

Body:
Linux manual page of operator define as same priority for both pre/post increment.

! ~ ++ -- + - (type) * & sizeof      right to left

post increment rule is first assigned and then increment

int j = i++;// here first whatever i value is there that will be assigned to j

Pre increment rule is first increment and then assign

int j = ++i;//++i itself will change i value & then modfied value will assign to j.

for e.g consider below example

#include<stdio.h>
int main()
{
        int x = 10;
        int *p = &x;// assume p holds 0x100
        printf("x = %d *p = %d p = %p\n",x,*p,p);
        ++*p++;
        /** in above statement ++ and * having same precedence,then you should check assocaitivity which is R to L
          So start solving from Right to Left, whichever operator came first , solve that one first
          first *p++ came , again solve p++ first, which is post increment,so address will be same
         *0x100  means = 10
         now ++10 means = 11
         **/
        printf("x = %d *p = %d p = %p\n",x,*p,p);
}


问:
Converting List of object to Guava Table data structure using lambda
I have list of ImmutableTriple object where for first and middle there could collection of last values (first, middle and last are triple values).
Now in order to make it queryable, I need to convert it to Guava Table data structure. I am able to achieve this, with for loop as below but I am wondering if I can achieve this more functionally using lambda expression.
here is the piece code -</p>
```<code>public static void main(String[] args) {
    //In real world, this list is coming from various transformation of lamda
    final List<ImmutableTriple<LocalDate, Integer, String>> list = ImmutableList.of(
            ImmutableTriple.of(LocalDate.now(), 1, "something"),
            ImmutableTriple.of(LocalDate.now(), 1, "anotherThing")
    );
    Table<LocalDate, Integer, List<String>> table = HashBasedTable.create();
    //is it possible to avoid this forEach and use side effect free lambda.
    list.forEach(s -> {
        final List<String> strings = table.get(s.left, s.middle);
        final List<String> slotList = strings == null ? new ArrayList<>() : strings;
        slotList.add(s.right);
        table.put(s.left, s.middle, slotList);
    });
    System.out.println(table);
}
</code>```

答:
There is a Tables class which contains a Collector to get your desired result.

```<code>Table<LocalDate, Integer, ImmutableList<String>> collect = list.stream()
        .collect(Tables.toTable(
                it -> it.left,
                it -> it.middle,
                it -> ImmutableList.of(it.right),
                (l1, l2) -> ImmutableList.<String>builder()
                        .addAll(l1).addAll(l2).build(),
                HashBasedTable::create));
</code>```

If you really want a mutable List then you can use:

```<code>Table<LocalDate, Integer, List<String>> collect = list.stream()
        .collect(Tables.toTable(
                it -> it.left,
                it -> it.middle,
                it -> Lists.newArrayList(it.right),
                (l1, l2) -> {l1.addAll(l2); return l1;},
                HashBasedTable::create));
</code>```

ContentLicense: CC BY-SA 4.0

Body:
I have list of ImmutableTriple object where for first and middle there could collection of last values (first, middle and last are triple values).
Now in order to make it queryable, I need to convert it to Guava Table data structure. I am able to achieve this, with for loop as below but I am wondering if I can achieve this more functionally using lambda expression.
here is the piece code -

public static void main(String[] args) {
    //In real world, this list is coming from various transformation of lamda
    final List<ImmutableTriple<LocalDate, Integer, String>> list = ImmutableList.of(
            ImmutableTriple.of(LocalDate.now(), 1, "something"),
            ImmutableTriple.of(LocalDate.now(), 1, "anotherThing")
    );
    Table<LocalDate, Integer, List<String>> table = HashBasedTable.create();
    //is it possible to avoid this forEach and use side effect free lambda.
    list.forEach(s -> {
        final List<String> strings = table.get(s.left, s.middle);
        final List<String> slotList = strings == null ? new ArrayList<>() : strings;
        slotList.add(s.right);
        table.put(s.left, s.middle, slotList);
    });
    System.out.println(table);
}

Body:
There is a Tables class which contains a Collector to get your desired result.

Table<LocalDate, Integer, ImmutableList<String>> collect = list.stream()
        .collect(Tables.toTable(
                it -> it.left,
                it -> it.middle,
                it -> ImmutableList.of(it.right),
                (l1, l2) -> ImmutableList.<String>builder()
                        .addAll(l1).addAll(l2).build(),
                HashBasedTable::create));

If you really want a mutable List then you can use:

Table<LocalDate, Integer, List<String>> collect = list.stream()
        .collect(Tables.toTable(
                it -> it.left,
                it -> it.middle,
                it -> Lists.newArrayList(it.right),
                (l1, l2) -> {l1.addAll(l2); return l1;},
                HashBasedTable::create));


问:
Regex to match "maybe valid month"
Basically I want to match the following:

```<code>const regex = /(?=.*[0-1])(?=.*[^0{2}])|(0[1-9]|1[012])$/
console.log('0, should be True: ', regex.test('0'))
console.log('1, should be True: ', regex.test('1'))
console.log('00, should be False: ', regex.test('00'))
console.log('01, should be True: ', regex.test('01'))
console.log('12, should be True: ', regex.test('12'))
console.log('99, should be False: ', regex.test('99'))
</code>```

Valid: 0, 1, 01-09, 10-12
Invalid: 00, >12

Anyone know how to do this?

Title: Regex to match "maybe valid month"

Body:
Basically I want to match the following:

const regex = /(?=.*[0-1])(?=.*[^0{2}])|(0[1-9]|1[012])$/
console.log('0, should be True: ', regex.test('0'))
console.log('1, should be True: ', regex.test('1'))
console.log('00, should be False: ', regex.test('00'))
console.log('01, should be True: ', regex.test('01'))
console.log('12, should be True: ', regex.test('12'))
console.log('99, should be False: ', regex.test('99'))

Valid: 0, 1, 01-09, 10-12
Invalid: 00, >12

Anyone know how to do this?

Body:
This will do:

/^0[1-9]|[0-9]|1[0-2]$/