问:
Adding header for HttpURLConnection
I'm trying to add header for my request using HttpUrlConnection but the method setRequestProperty() doesn't seem working. The server side doesn't receive any request with my header.

```<code>HttpURLConnection hc;
    try {
        String authorization = "";
        URL address = new URL(url);
        hc = (HttpURLConnection) address.openConnection();

        hc.setDoOutput(true);
        hc.setDoInput(true);
        hc.setUseCaches(false);

        if (username != null && password != null) {
            authorization = username + ":" + password;
        }

        if (authorization != null) {
            byte[] encodedBytes;
            encodedBytes = Base64.encode(authorization.getBytes(), 0);
            authorization = "Basic " + encodedBytes;
            hc.setRequestProperty("Authorization", authorization);
        }
</code>```

答:
If you are using Java 8, use the code below.</p>

```<code>URLConnection connection = url.openConnection();
HttpURLConnection httpConn = (HttpURLConnection) connection;

String basicAuth = Base64.getEncoder().encodeToString((username+":"+password).getBytes(StandardCharsets.UTF_8));
httpConn.setRequestProperty ("Authorization", "Basic "+basicAuth);
</code>```

Title: Adding header for HttpURLConnection

Body:
I'm trying to add header for my request using HttpUrlConnection but the method setRequestProperty() doesn't seem working. The server side doesn't receive any request with my header.

HttpURLConnection hc;
    try {
        String authorization = "";
        URL address = new URL(url);
        hc = (HttpURLConnection) address.openConnection();

        hc.setDoOutput(true);
        hc.setDoInput(true);
        hc.setUseCaches(false);

        if (username != null && password != null) {
            authorization = username + ":" + password;
        }

        if (authorization != null) {
            byte[] encodedBytes;
            encodedBytes = Base64.encode(authorization.getBytes(), 0);
            authorization = "Basic " + encodedBytes;
            hc.setRequestProperty("Authorization", authorization);
        }

Body:
If you are using Java 8, use the code below.

URLConnection connection = url.openConnection();
HttpURLConnection httpConn = (HttpURLConnection) connection;

String basicAuth = Base64.getEncoder().encodeToString((username+":"+password).getBytes(StandardCharsets.UTF_8));
httpConn.setRequestProperty ("Authorization", "Basic "+basicAuth);


来源: stackexchange

问:
Accessing Microsoft SQL Server using Windows authentication on a Mac in Python
I am having trouble figuring out the configuration steps and code that I need to access a SQL Server database via Windows Authentication from a Mac.

On the Windows machine I can use the following code:

```<code>import getpass

import pypyodbc

cnxn_string = (r"DRIVER={ODBC Driver 13 for SQL Server};"
               r"SERVER=[Host]\[Instance];"
               r"DATABASE=[Database]"
               r"UID=[Domain]\[Username];"
               r"Trusted_Connection=yes;"% (getpass.getuser()))

cnxn = pypyodbc.connect(cnxn_string)
</code>```

But on a Mac I believe I need to re-enter my username and password.

How can I create a similar python connection to the SQL Server, via Windows Authentication, on my mac? I am open to any python modules (it does not have to be pypyodbc).

答:
Using FreeTDS + pymssql it's pretty trivial to connect.

Note: I would recommend using FreeTDS 1.0+ and pymssql 2.2.0+ because there are compatibility issues with FreeTDS 1.0 and pymssql 2.1.3.

Here's an example using pymssql:

```<code>import pymssql
pymssql.connect(server="myserver\\myinstance",
                user="mydomain\\myuser",
                password="mypassword",
                database="mydatabase")
</code>```

Note: the instance is parsed from the server using the backslash character, but we need to escape this because this is also the escape character. The same can be said for parsing the Active Directory domain from the user.

ContentLicense: CC BY-SA 3.0

Body:
I am having trouble figuring out the configuration steps and code that I need to access a SQL Server database via Windows Authentication from a Mac.

On the Windows machine I can use the following code:

import getpass

import pypyodbc

cnxn_string = (r"DRIVER={ODBC Driver 13 for SQL Server};"
               r"SERVER=[Host]\[Instance];"
               r"DATABASE=[Database]"
               r"UID=[Domain]\[Username];"
               r"Trusted_Connection=yes;"% (getpass.getuser()))

cnxn = pypyodbc.connect(cnxn_string)

But on a Mac I believe I need to re-enter my username and password.

How can I create a similar python connection to the SQL Server, via Windows Authentication, on my mac? I am open to any python modules (it does not have to be pypyodbc).

Body:
Using FreeTDS + pymssql it's pretty trivial to connect.

Note: I would recommend using FreeTDS 1.0+ and pymssql 2.2.0+ because there are compatibility issues with FreeTDS 1.0 and pymssql 2.1.3.

Here's an example using pymssql:

import pymssql
pymssql.connect(server="myserver\\myinstance",
                user="mydomain\\myuser",
                password="mypassword",
                database="mydatabase")

Note: the instance is parsed from the server using the backslash character, but we need to escape this because this is also the escape character. The same can be said for parsing the Active Directory domain from the user.


问:
Toolbar on windows xp and vista taskbar
I have made a toolbar that I want to enable from a systray application written in C#, the actual toolbar enabling is done from a C++ part using [DLLImport].

Current I use:

```<code>SHLoadInProc(__uuidof(MyBandLoader))
</code>```

but this fails on vista (SHLoadInProc is not implemented any more), and on Windows XP SP2 with IE6 (the quick launch toolbar vanishes after reboot).

On Vista I have tried to with: CocreateInstance()  and BandSite->AddBand(), but using the guid of the toolbar dll gave me either a segmentation fault or the address bar.

Is there another way to enable a toolbar from another program on XP and Vista?

答:
On Vista there's a new poorly-documented interface called ITrayDeskBand.

Create an instance of this via CoCreateInstance, and then call ShowDeskBand([CLSID of your toolbar]) on the returned pointer (in C++ - I'm not sure how you create all the relevant bits for PInvoke in C# - might be easier to write a simple C++ dll to expose this function)

That only works on Vista though, on XP you need to continue with the SHLoadInProc method above, so you need to test the OS version and do the appropriate thing.

Be careful if you're lifting code from that codeproject article - it's full of subtle bugs, although many of them are discussed in the comments

OwnerDisplayName: Eirik Nygaard

ContentLicense: CC BY-SA 2.5

Title: Toolbar on windows xp and vista taskbar

Body:
I have made a toolbar that I want to enable from a systray application written in C#, the actual toolbar enabling is done from a C++ part using [DLLImport].

Current I use:

SHLoadInProc(__uuidof(MyBandLoader))

but this fails on vista (SHLoadInProc is not implemented any more), and on Windows XP SP2 with IE6 (the quick launch toolbar vanishes after reboot).

On Vista I have tried to with: CocreateInstance()  and BandSite->AddBand(), but using the guid of the toolbar dll gave me either a segmentation fault or the address bar.

Is there another way to enable a toolbar from another program on XP and Vista?


问:
How to calculate the data transfer rate of a particular site(virtual host) in AWS
I have virtual host files(sites) setup on two linux EC2 instances which are behind an ELB. I would like to know the data transfer rate of one particular site which is hosted on these EC2 instances. There are almost 30 virtual hosts on each EC2 instance and I need to calculate the average data transfer rate of all these sites. From cloudwatch I could only gather the information at service level but not for particular site. Is there any way to accomplish this?</p>

答:
In this case, I recommend 2 things:

Do not use ELB or EC2 directly for data transfer.

If you want to know the exactly MB/GB/TB:

Work with CloudFront in front of your ELB.

Register one CloudFront distribution for each site (domain, subdomain or whatever).

If you do that, you will have more control and save money (data transfer) but it depends on the region (sometimes it can be a bit more expensive).


问:
Fast sorting/Filtering based on alternating values
Consider the following sample data table,

```<code>dt <- data.table(src = LETTERS[1:10],
                 dst = LETTERS[10:1],
                 src1 = letters[15:24],
                 dst1 = letters[24:15])

#which looks like,
#    src dst src1 dst1
# 1:   A   J    o    x
# 2:   B   I    p    w
# 3:   C   H    q    v
# 4:   D   G    r    u
# 5:   E   F    s    t
# 6:   F   E    t    s
# 7:   G   D    u    r
# 8:   H   C    v    q
# 9:   I   B    w    p
#10:   J   A    x    o
</code>```

The first goal is to order it based on reversed rowwise paired elements (src - dst & src1 - dst1) which can be achieved as follows to create the 5 'pairs':

```<code>dt[, key := paste0(pmin(src, dst), pmax(src, dst), pmin(src1, dst1), pmax(src1, dst1))][order(key)]

#    src dst src1 dst1  key
# 1:   A   J    o    x AJox
# 2:   J   A    x    o AJox
# 3:   B   I    p    w BIpw
# 4:   I   B    w    p BIpw
# 5:   C   H    q    v CHqv
# 6:   H   C    v    q CHqv
# 7:   D   G    r    u DGru
# 8:   G   D    u    r DGru
# 9:   E   F    s    t EFst
#10:   F   E    t    s EFst
</code>```

However, in real life there could be at least one row that does not have a pair ("incomplete flow"). So a real life example of that would be,

```<code>#              cdatetime   srcaddr dstaddr  srcport   dstport     key    totals time_diff
# 1: 2017-05-12 14:58:32    IP_1    IP_2   54793    8080 182808054793      3        NA
# 2: 2017-05-12 14:58:32    IP_2    IP_1    8080   54793 182808054793      3         0
# 3: 2017-05-17 08:37:16    IP_1    IP_2   54793    8080 182808054793      3    409124
# 4: 2017-05-11 08:12:28    IP_1    IP_2   54813    8080 182808054813      3        NA
# 5: 2017-05-11 08:12:28    IP_2    IP_1    8080   54813 182808054813      3         0
# 6: 2017-05-17 08:37:16    IP_1    IP_2   54813    8080 182808054813      3    519888
# 7: 2017-05-02 06:51:16    IP_1    IP_2   50794    8080 182808050794      5        NA
# 8: 2017-05-02 06:51:16    IP_2    IP_1    8080   50794 182808050794      5         0
# 9: 2017-05-08 06:57:08    IP_1    IP_2   50794    8080 182808050794      5    518752
#10: 2017-05-11 06:32:49    IP_1    IP_2   50794    8080 182808050794      5    257741
#11: 2017-05-11 06:32:49    IP_2    IP_1    8080   50794 182808050794      5         0
#12: 2017-05-04 06:52:05    IP_1    IP_2   51896    8080 182808051896      5        NA
#13: 2017-05-04 06:52:05    IP_2    IP_1    8080   51896 182808051896      5         0
#14: 2017-05-04 10:22:26    IP_1    IP_2   51896    8080 182808051896      5     12621
#15: 2017-05-04 10:22:26    IP_2    IP_1    8080   51896 182808051896      5         0
#16: 2017-05-08 07:22:47    IP_1    IP_2   51896    8080 182808051896      5    334821
#17: 2017-05-15 05:56:00    IP_1    IP_2   62744     162  17016262744      3        NA
#18: 2017-05-17 10:41:00    IP_1    IP_2   62744     162  17016262744      3    189900
#19: 2017-05-18 09:31:00    IP_1    IP_2   62744     162  17016262744      3     82200
</code>```

The second goal now is to remove those incomplete flows. Now, to identify those "incomplete flows" we calculate the time difference between them and set 30 as the threshold. The tricky part comes here; If we have only 2 rows and they are more than 30 seconds apart, then filter them both out and for 3 rows (for a certain key) then the one with time diff > 30 needs to go - or if two of them have  > 30 seconds apart, remove all 3. However, when we have 4 or more rows, then we need to drop the one(s) with time difference > 30 that do not have another pair with < 30. From the above table, rows 3, 6, 9, 16, 17, 18, 19 have to be removed based on the condition that they are more than 30 seconds apart. Looking at cdatetime will clarify which are the complete flows.

Expected output would be,

```<code> #        cdatetime      srcaddr dstaddr srcport dstport          key totals time_diff
# 1: 2017-05-12 14:58:32    IP_1    IP_2   54793    8080 182808054793      3        NA
# 2: 2017-05-12 14:58:32    IP_2    IP_1    8080   54793 182808054793      3         0
# 3: 2017-05-11 08:12:28    IP_1    IP_2   54813    8080 182808054813      3        NA
# 4: 2017-05-11 08:12:28    IP_2    IP_1    8080   54813 182808054813      3         0
# 5: 2017-05-02 06:51:16    IP_1    IP_2   50794    8080 182808050794      5        NA
# 6: 2017-05-02 06:51:16    IP_2    IP_1    8080   50794 182808050794      5         0
# 7: 2017-05-11 06:32:49    IP_1    IP_2   50794    8080 182808050794      5    257741
# 8: 2017-05-11 06:32:49    IP_2    IP_1    8080   50794 182808050794      5         0
# 9: 2017-05-04 06:52:05    IP_1    IP_2   51896    8080 182808051896      5        NA
#10: 2017-05-04 06:52:05    IP_2    IP_1    8080   51896 182808051896      5         0
#11: 2017-05-04 10:22:26    IP_1    IP_2   51896    8080 182808051896      5     12621
#12: 2017-05-04 10:22:26    IP_2    IP_1    8080   51896 182808051896      5         0
</code>```

DATA OF REAL LIFE EXAMPLES ABOVE

```<code>structure(list(cdatetime = structure(c(1494590312, 1494590312,
1494999436, 1494479548, 1494479548, 1494999436, 1493697076, 1493697076,
1494215828, 1494473569, 1494473569, 1493869925, 1493869925, 1493882546,
1493882546, 1494217367, 1494816960, 1495006860, 1495089060), class = c("POSIXct",
"POSIXt"), tzone = ""), srcaddr = structure(c(1L, 2L, 1L, 1L,
2L, 1L, 1L, 2L, 1L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 1L, 1L, 1L), .Label = c("IP_1",
"IP_2"), class = "factor"), dstaddr = structure(c(2L, 1L, 2L,
2L, 1L, 2L, 2L, 1L, 2L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 2L, 2L, 2L
), .Label = c("IP_1", "IP_2"), class = "factor"), srcport = c(54793L,
8080L, 54793L, 54813L, 8080L, 54813L, 50794L, 8080L, 50794L,
50794L, 8080L, 51896L, 8080L, 51896L, 8080L, 51896L, 62744L,
62744L, 62744L), dstport = c(8080L, 54793L, 8080L, 8080L, 54813L,
8080L, 8080L, 50794L, 8080L, 8080L, 50794L, 8080L, 51896L, 8080L,
51896L, 8080L, 162L, 162L, 162L), key = c(182808054793, 182808054793,
182808054793, 182808054813, 182808054813, 182808054813, 182808050794,
182808050794, 182808050794, 182808050794, 182808050794, 182808051896,
182808051896, 182808051896, 182808051896, 182808051896, 17016262744,
17016262744, 17016262744), totals = c(3L, 3L, 3L, 3L, 3L, 3L,
5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 3L, 3L, 3L), time_diff = c(NA,
0, 409124, NA, 0, 519888, NA, 0, 518752, 257741, 0, NA, 0, 12621,
0, 334821, NA, 189900, 82200)), .Names = c("cdatetime", "srcaddr",
"dstaddr", "srcport", "dstport", "key", "totals", "time_diff"
), row.names = c(NA, -19L), class = c("data.table", "data.frame"
))
</code>```

All the above will run on a dataSet with approx. 180M rows, so efficiency is the key word here.

答:
This is my suggestion to identify pairs of complete flows. (However, note the caveats at the end)

```<code>library(data.table)   # CRAN version 1.10.4 used

# set keys to avoid using order() repeatedly
setkey(DT, key, cdatetime)
# compute time diff again, grouped by key,
# using shift() with default "lag" type and a useful fill value
DT[, time_diff := difftime(cdatetime, shift(cdatetime, fill = -Inf), units = "secs"), by = key]
# now find the begin of a potentially new pair where time_diff > 30 secs
# and count the jumps within each key group using cumsum()
DT[, pair.id := cumsum(time_diff > 30), by = key]
# count the number of partners within each supposedly pair
DT[, count.pairs := .N, .(key, pair.id)]
</code>```

Note that in R, the logical values can be coerced to numeric:

```<code>as.integer(FALSE)
#[1] 0
as.integer(TRUE)
#[1] 1
</code>```

So, everytime cumsum() finds time_diff > 30 to be TRUE the pair.id count is advanced by one. Otherwise, i.e., if the time differences is less than 30 secs, the pair.id remains the same. By this, pairs or groups of events laying in the 30 secs time window are identified.

Now, DThas gained two additional columns (note that setkey()has changed the order of rows):

```<code>              cdatetime srcaddr dstaddr srcport dstport          key totals   time_diff pair.id count.pairs
 1: 2017-05-15 04:56:00    IP_1    IP_2   62744     162  17016262744      3    Inf secs       1           1
 2: 2017-05-17 09:41:00    IP_1    IP_2   62744     162  17016262744      3 189900 secs       2           1
 3: 2017-05-18 08:31:00    IP_1    IP_2   62744     162  17016262744      3  82200 secs       3           1
 4: 2017-05-02 05:51:16    IP_1    IP_2   50794    8080 182808050794      5    Inf secs       1           2
 5: 2017-05-02 05:51:16    IP_2    IP_1    8080   50794 182808050794      5      0 secs       1           2
 6: 2017-05-08 05:57:08    IP_1    IP_2   50794    8080 182808050794      5 518752 secs       2           1
 7: 2017-05-11 05:32:49    IP_1    IP_2   50794    8080 182808050794      5 257741 secs       3           2
 8: 2017-05-11 05:32:49    IP_2    IP_1    8080   50794 182808050794      5      0 secs       3           2
 9: 2017-05-04 05:52:05    IP_1    IP_2   51896    8080 182808051896      5    Inf secs       1           2
10: 2017-05-04 05:52:05    IP_2    IP_1    8080   51896 182808051896      5      0 secs       1           2
11: 2017-05-04 09:22:26    IP_1    IP_2   51896    8080 182808051896      5  12621 secs       2           2
12: 2017-05-04 09:22:26    IP_2    IP_1    8080   51896 182808051896      5      0 secs       2           2
13: 2017-05-08 06:22:47    IP_1    IP_2   51896    8080 182808051896      5 334821 secs       3           1
14: 2017-05-12 13:58:32    IP_1    IP_2   54793    8080 182808054793      3    Inf secs       1           2
15: 2017-05-12 13:58:32    IP_2    IP_1    8080   54793 182808054793      3      0 secs       1           2
16: 2017-05-17 07:37:16    IP_1    IP_2   54793    8080 182808054793      3 409124 secs       2           1
17: 2017-05-11 07:12:28    IP_1    IP_2   54813    8080 182808054813      3    Inf secs       1           2
18: 2017-05-11 07:12:28    IP_2    IP_1    8080   54813 182808054813      3      0 secs       1           2
19: 2017-05-17 07:37:16    IP_1    IP_2   54813    8080 182808054813      3 519888 secs       2           1
</code>```

Now, it has to be decided which rows to keep. In this small sample there are only groups with two memebers (pairs) or singles.

```<code>DT[count.pairs > 1]
</code>```

shows only pairs

```<code>              cdatetime srcaddr dstaddr srcport dstport          key totals   time_diff pair.id count.pairs
 1: 2017-05-02 05:51:16    IP_1    IP_2   50794    8080 182808050794      5    Inf secs       1           2
 2: 2017-05-02 05:51:16    IP_2    IP_1    8080   50794 182808050794      5      0 secs       1           2
 3: 2017-05-11 05:32:49    IP_1    IP_2   50794    8080 182808050794      5 257741 secs       3           2
 4: 2017-05-11 05:32:49    IP_2    IP_1    8080   50794 182808050794      5      0 secs       3           2
 5: 2017-05-04 05:52:05    IP_1    IP_2   51896    8080 182808051896      5    Inf secs       1           2
 6: 2017-05-04 05:52:05    IP_2    IP_1    8080   51896 182808051896      5      0 secs       1           2
 7: 2017-05-04 09:22:26    IP_1    IP_2   51896    8080 182808051896      5  12621 secs       2           2
 8: 2017-05-04 09:22:26    IP_2    IP_1    8080   51896 182808051896      5      0 secs       2           2
 9: 2017-05-12 13:58:32    IP_1    IP_2   54793    8080 182808054793      3    Inf secs       1           2
10: 2017-05-12 13:58:32    IP_2    IP_1    8080   54793 182808054793      3      0 secs       1           2
11: 2017-05-11 07:12:28    IP_1    IP_2   54813    8080 182808054813      3    Inf secs       1           2
12: 2017-05-11 07:12:28    IP_2    IP_1    8080   54813 182808054813      3      0 secs       1           2
</code>```

while

```<code>DT[count.pairs <= 1]
</code>```

shows the singles to be removed:

```<code>             cdatetime srcaddr dstaddr srcport dstport          key totals   time_diff pair.id count.pairs
1: 2017-05-15 04:56:00    IP_1    IP_2   62744     162  17016262744      3    Inf secs       1           1
2: 2017-05-17 09:41:00    IP_1    IP_2   62744     162  17016262744      3 189900 secs       2           1
3: 2017-05-18 08:31:00    IP_1    IP_2   62744     162  17016262744      3  82200 secs       3           1
4: 2017-05-08 05:57:08    IP_1    IP_2   50794    8080 182808050794      5 518752 secs       2           1
5: 2017-05-08 06:22:47    IP_1    IP_2   51896    8080 182808051896      5 334821 secs       3           1
6: 2017-05-17 07:37:16    IP_1    IP_2   54793    8080 182808054793      3 409124 secs       2           1
7: 2017-05-17 07:37:16    IP_1    IP_2   54813    8080 182808054813      3 519888 secs       2           1
</code>```

Caveat It might happen in production data that more than two events with the same key fall within a 30 seconds timespan. There are some options to deal with this:

Keep only exact pairs: DT[count.pairs == 2] ignoring all other cases.

Keep only "pairs" with an even number of partners: DT[count.pairs %% 2L == 0L] (although this may keep cases where the number of inbound connections does not equal the number of outbound connections)

and other options which would require additional work, e.g., separately counting of inbound and outbound connections.

Finally, the 30 secs time window could be decreased.

The statistics can be created using

```<code>DT[, .N, count.pairs]
#   count.pairs  N
#1:           1  7
#2:           2 12
</code>```

It would be interesting to see this for the production data set.

Body:
Consider the following sample data table,

dt <- data.table(src = LETTERS[1:10],
                 dst = LETTERS[10:1],
                 src1 = letters[15:24],
                 dst1 = letters[24:15])

#which looks like,
#    src dst src1 dst1
# 1:   A   J    o    x
# 2:   B   I    p    w
# 3:   C   H    q    v
# 4:   D   G    r    u
# 5:   E   F    s    t
# 6:   F   E    t    s
# 7:   G   D    u    r
# 8:   H   C    v    q
# 9:   I   B    w    p
#10:   J   A    x    o

The first goal is to order it based on reversed rowwise paired elements (src - dst & src1 - dst1) which can be achieved as follows to create the 5 'pairs':

dt[, key := paste0(pmin(src, dst), pmax(src, dst), pmin(src1, dst1), pmax(src1, dst1))][order(key)]

#    src dst src1 dst1  key
# 1:   A   J    o    x AJox
# 2:   J   A    x    o AJox
# 3:   B   I    p    w BIpw
# 4:   I   B    w    p BIpw
# 5:   C   H    q    v CHqv
# 6:   H   C    v    q CHqv
# 7:   D   G    r    u DGru
# 8:   G   D    u    r DGru
# 9:   E   F    s    t EFst
#10:   F   E    t    s EFst

However, in real life there could be at least one row that does not have a pair ("incomplete flow"). So a real life example of that would be,

#              cdatetime   srcaddr dstaddr  srcport   dstport     key    totals time_diff
# 1: 2017-05-12 14:58:32    IP_1    IP_2   54793    8080 182808054793      3        NA
# 2: 2017-05-12 14:58:32    IP_2    IP_1    8080   54793 182808054793      3         0
# 3: 2017-05-17 08:37:16    IP_1    IP_2   54793    8080 182808054793      3    409124
# 4: 2017-05-11 08:12:28    IP_1    IP_2   54813    8080 182808054813      3        NA
# 5: 2017-05-11 08:12:28    IP_2    IP_1    8080   54813 182808054813      3         0
# 6: 2017-05-17 08:37:16    IP_1    IP_2   54813    8080 182808054813      3    519888
# 7: 2017-05-02 06:51:16    IP_1    IP_2   50794    8080 182808050794      5        NA
# 8: 2017-05-02 06:51:16    IP_2    IP_1    8080   50794 182808050794      5         0
# 9: 2017-05-08 06:57:08    IP_1    IP_2   50794    8080 182808050794      5    518752
#10: 2017-05-11 06:32:49    IP_1    IP_2   50794    8080 182808050794      5    257741
#11: 2017-05-11 06:32:49    IP_2    IP_1    8080   50794 182808050794      5         0
#12: 2017-05-04 06:52:05    IP_1    IP_2   51896    8080 182808051896      5        NA
#13: 2017-05-04 06:52:05    IP_2    IP_1    8080   51896 182808051896      5         0
#14: 2017-05-04 10:22:26    IP_1    IP_2   51896    8080 182808051896      5     12621
#15: 2017-05-04 10:22:26    IP_2    IP_1    8080   51896 182808051896      5         0
#16: 2017-05-08 07:22:47    IP_1    IP_2   51896    8080 182808051896      5    334821
#17: 2017-05-15 05:56:00    IP_1    IP_2   62744     162  17016262744      3        NA
#18: 2017-05-17 10:41:00    IP_1    IP_2   62744     162  17016262744      3    189900
#19: 2017-05-18 09:31:00    IP_1    IP_2   62744     162  17016262744      3     82200

The second goal now is to remove those incomplete flows. Now, to identify those "incomplete flows" we calculate the time difference between them and set 30 as the threshold. The tricky part comes here; If we have only 2 rows and they are more than 30 seconds apart, then filter them both out and for 3 rows (for a certain key) then the one with time diff > 30 needs to go - or if two of them have  > 30 seconds apart, remove all 3. However, when we have 4 or more rows, then we need to drop the one(s) with time difference > 30 that do not have another pair with < 30. From the above table, rows 3, 6, 9, 16, 17, 18, 19 have to be removed based on the condition that they are more than 30 seconds apart. Looking at cdatetime will clarify which are the complete flows.

Expected output would be,

 #        cdatetime      srcaddr dstaddr srcport dstport          key totals time_diff
# 1: 2017-05-12 14:58:32    IP_1    IP_2   54793    8080 182808054793      3        NA
# 2: 2017-05-12 14:58:32    IP_2    IP_1    8080   54793 182808054793      3         0
# 3: 2017-05-11 08:12:28    IP_1    IP_2   54813    8080 182808054813      3        NA
# 4: 2017-05-11 08:12:28    IP_2    IP_1    8080   54813 182808054813      3         0
# 5: 2017-05-02 06:51:16    IP_1    IP_2   50794    8080 182808050794      5        NA
# 6: 2017-05-02 06:51:16    IP_2    IP_1    8080   50794 182808050794      5         0
# 7: 2017-05-11 06:32:49    IP_1    IP_2   50794    8080 182808050794      5    257741
# 8: 2017-05-11 06:32:49    IP_2    IP_1    8080   50794 182808050794      5         0
# 9: 2017-05-04 06:52:05    IP_1    IP_2   51896    8080 182808051896      5        NA
#10: 2017-05-04 06:52:05    IP_2    IP_1    8080   51896 182808051896      5         0
#11: 2017-05-04 10:22:26    IP_1    IP_2   51896    8080 182808051896      5     12621
#12: 2017-05-04 10:22:26    IP_2    IP_1    8080   51896 182808051896      5         0

DATA OF REAL LIFE EXAMPLES ABOVE

structure(list(cdatetime = structure(c(1494590312, 1494590312,
1494999436, 1494479548, 1494479548, 1494999436, 1493697076, 1493697076,
1494215828, 1494473569, 1494473569, 1493869925, 1493869925, 1493882546,
1493882546, 1494217367, 1494816960, 1495006860, 1495089060), class = c("POSIXct",
"POSIXt"), tzone = ""), srcaddr = structure(c(1L, 2L, 1L, 1L,
2L, 1L, 1L, 2L, 1L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 1L, 1L, 1L), .Label = c("IP_1",
"IP_2"), class = "factor"), dstaddr = structure(c(2L, 1L, 2L,
2L, 1L, 2L, 2L, 1L, 2L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 2L, 2L, 2L
), .Label = c("IP_1", "IP_2"), class = "factor"), srcport = c(54793L,
8080L, 54793L, 54813L, 8080L, 54813L, 50794L, 8080L, 50794L,
50794L, 8080L, 51896L, 8080L, 51896L, 8080L, 51896L, 62744L,
62744L, 62744L), dstport = c(8080L, 54793L, 8080L, 8080L, 54813L,
8080L, 8080L, 50794L, 8080L, 8080L, 50794L, 8080L, 51896L, 8080L,
51896L, 8080L, 162L, 162L, 162L), key = c(182808054793, 182808054793,
182808054793, 182808054813, 182808054813, 182808054813, 182808050794,
182808050794, 182808050794, 182808050794, 182808050794, 182808051896,
182808051896, 182808051896, 182808051896, 182808051896, 17016262744,
17016262744, 17016262744), totals = c(3L, 3L, 3L, 3L, 3L, 3L,
5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 3L, 3L, 3L), time_diff = c(NA,
0, 409124, NA, 0, 519888, NA, 0, 518752, 257741, 0, NA, 0, 12621,
0, 334821, NA, 189900, 82200)), .Names = c("cdatetime", "srcaddr",
"dstaddr", "srcport", "dstport", "key", "totals", "time_diff"
), row.names = c(NA, -19L), class = c("data.table", "data.frame"
))

All the above will run on a dataSet with approx. 180M rows, so efficiency is the key word here.

Body:
This is my suggestion to identify pairs of complete flows. (However, note the caveats at the end)

library(data.table)   # CRAN version 1.10.4 used

# set keys to avoid using order() repeatedly
setkey(DT, key, cdatetime)
# compute time diff again, grouped by key,
# using shift() with default "lag" type and a useful fill value
DT[, time_diff := difftime(cdatetime, shift(cdatetime, fill = -Inf), units = "secs"), by = key]
# now find the begin of a potentially new pair where time_diff > 30 secs
# and count the jumps within each key group using cumsum()
DT[, pair.id := cumsum(time_diff > 30), by = key]
# count the number of partners within each supposedly pair
DT[, count.pairs := .N, .(key, pair.id)]

Note that in R, the logical values can be coerced to numeric:

as.integer(FALSE)
#[1] 0
as.integer(TRUE)
#[1] 1

So, everytime cumsum() finds time_diff > 30 to be TRUE the pair.id count is advanced by one. Otherwise, i.e., if the time differences is less than 30 secs, the pair.id remains the same. By this, pairs or groups of events laying in the 30 secs time window are identified.

Now, DThas gained two additional columns (note that setkey()has changed the order of rows):

              cdatetime srcaddr dstaddr srcport dstport          key totals   time_diff pair.id count.pairs
 1: 2017-05-15 04:56:00    IP_1    IP_2   62744     162  17016262744      3    Inf secs       1           1
 2: 2017-05-17 09:41:00    IP_1    IP_2   62744     162  17016262744      3 189900 secs       2           1
 3: 2017-05-18 08:31:00    IP_1    IP_2   62744     162  17016262744      3  82200 secs       3           1
 4: 2017-05-02 05:51:16    IP_1    IP_2   50794    8080 182808050794      5    Inf secs       1           2
 5: 2017-05-02 05:51:16    IP_2    IP_1    8080   50794 182808050794      5      0 secs       1           2
 6: 2017-05-08 05:57:08    IP_1    IP_2   50794    8080 182808050794      5 518752 secs       2           1
 7: 2017-05-11 05:32:49    IP_1    IP_2   50794    8080 182808050794      5 257741 secs       3           2
 8: 2017-05-11 05:32:49    IP_2    IP_1    8080   50794 182808050794      5      0 secs       3           2
 9: 2017-05-04 05:52:05    IP_1    IP_2   51896    8080 182808051896      5    Inf secs       1           2
10: 2017-05-04 05:52:05    IP_2    IP_1    8080   51896 182808051896      5      0 secs       1           2
11: 2017-05-04 09:22:26    IP_1    IP_2   51896    8080 182808051896      5  12621 secs       2           2
12: 2017-05-04 09:22:26    IP_2    IP_1    8080   51896 182808051896      5      0 secs       2           2
13: 2017-05-08 06:22:47    IP_1    IP_2   51896    8080 182808051896      5 334821 secs       3           1
14: 2017-05-12 13:58:32    IP_1    IP_2   54793    8080 182808054793      3    Inf secs       1           2
15: 2017-05-12 13:58:32    IP_2    IP_1    8080   54793 182808054793      3      0 secs       1           2
16: 2017-05-17 07:37:16    IP_1    IP_2   54793    8080 182808054793      3 409124 secs       2           1
17: 2017-05-11 07:12:28    IP_1    IP_2   54813    8080 182808054813      3    Inf secs       1           2
18: 2017-05-11 07:12:28    IP_2    IP_1    8080   54813 182808054813      3      0 secs       1           2
19: 2017-05-17 07:37:16    IP_1    IP_2   54813    8080 182808054813      3 519888 secs       2           1

Now, it has to be decided which rows to keep. In this small sample there are only groups with two memebers (pairs) or singles.

DT[count.pairs > 1]

shows only pairs

              cdatetime srcaddr dstaddr srcport dstport          key totals   time_diff pair.id count.pairs
 1: 2017-05-02 05:51:16    IP_1    IP_2   50794    8080 182808050794      5    Inf secs       1           2
 2: 2017-05-02 05:51:16    IP_2    IP_1    8080   50794 182808050794      5      0 secs       1           2
 3: 2017-05-11 05:32:49    IP_1    IP_2   50794    8080 182808050794      5 257741 secs       3           2
 4: 2017-05-11 05:32:49    IP_2    IP_1    8080   50794 182808050794      5      0 secs       3           2
 5: 2017-05-04 05:52:05    IP_1    IP_2   51896    8080 182808051896      5    Inf secs       1           2
 6: 2017-05-04 05:52:05    IP_2    IP_1    8080   51896 182808051896      5      0 secs       1           2
 7: 2017-05-04 09:22:26    IP_1    IP_2   51896    8080 182808051896      5  12621 secs       2           2
 8: 2017-05-04 09:22:26    IP_2    IP_1    8080   51896 182808051896      5      0 secs       2           2
 9: 2017-05-12 13:58:32    IP_1    IP_2   54793    8080 182808054793      3    Inf secs       1           2
10: 2017-05-12 13:58:32    IP_2    IP_1    8080   54793 182808054793      3      0 secs       1           2
11: 2017-05-11 07:12:28    IP_1    IP_2   54813    8080 182808054813      3    Inf secs       1           2
12: 2017-05-11 07:12:28    IP_2    IP_1    8080   54813 182808054813      3      0 secs       1           2

while

DT[count.pairs <= 1]

shows the singles to be removed:

             cdatetime srcaddr dstaddr srcport dstport          key totals   time_diff pair.id count.pairs
1: 2017-05-15 04:56:00    IP_1    IP_2   62744     162  17016262744      3    Inf secs       1           1
2: 2017-05-17 09:41:00    IP_1    IP_2   62744     162  17016262744      3 189900 secs       2           1
3: 2017-05-18 08:31:00    IP_1    IP_2   62744     162  17016262744      3  82200 secs       3           1
4: 2017-05-08 05:57:08    IP_1    IP_2   50794    8080 182808050794      5 518752 secs       2           1
5: 2017-05-08 06:22:47    IP_1    IP_2   51896    8080 182808051896      5 334821 secs       3           1
6: 2017-05-17 07:37:16    IP_1    IP_2   54793    8080 182808054793      3 409124 secs       2           1
7: 2017-05-17 07:37:16    IP_1    IP_2   54813    8080 182808054813      3 519888 secs       2           1

Caveat It might happen in production data that more than two events with the same key fall within a 30 seconds timespan. There are some options to deal with this:

Keep only exact pairs: DT[count.pairs == 2] ignoring all other cases.

Keep only "pairs" with an even number of partners: DT[count.pairs %% 2L == 0L] (although this may keep cases where the number of inbound connections does not equal the number of outbound connections)

and other options which would require additional work, e.g., separately counting of inbound and outbound connections.

Finally, the 30 secs time window could be decreased.

The statistics can be created using

DT[, .N, count.pairs]
#   count.pairs  N
#1:           1  7
#2:           2 12

It would be interesting to see this for the production data set.


问:
Webservice failing on startup when built with Java 8
We are building a webservices with Java8 and deploying it to Tomcat8. We get following error during start up.

Caused by: org.springframework.beans.factory.BeanDefinitionStoreException: IOException parsing XML document from class path resource [META-INF/cxf/cxf-servlet.xml]; nested exception is java.io.FileNotFoundException: class path resource [META-INF/cxf/cxf-servlet.xml] cannot be opened because it does not exist

When we build the same component with Java7 we are not seeing this issue.

Here is our pom dependency

```<code>   <dependency>
        <groupId>org.apache.cxf</groupId>
        <artifactId>cxf-rt-frontend-jaxws</artifactId>
        <version>2.7.0</version>
    </dependency>
</code>```

Here is our web.xml

```<code><context-param>
    <param-name>contextConfigLocation</param-name>
    <param-value>classpath:applicationConfig.xml</param-value>
</context-param>

<listener>
    <listener-class>
        org.springframework.web.context.ContextLoaderListener
    </listener-class>
</listener>

<servlet>
    <servlet-name>CXFServlet</servlet-name>
    <display-name>CXF Servlet</display-name>
    <servlet-class>
        org.apache.cxf.transport.servlet.CXFServlet
    </servlet-class>
    <load-on-startup>1</load-on-startup>
</servlet>

<servlet-mapping>
    <servlet-name>CXFServlet</servlet-name>
    <url-pattern>/*</url-pattern>
</servlet-mapping>
<welcome-file-list>
    <welcome-file>index.jsp</welcome-file>
</welcome-file-list>
</code>```

ApplicationConfig.xml has the import

```<code><import resource="classpath:META-INF/cxf/cxf.xml" />
<import resource="classpath:META-INF/cxf/cxf-servlet.xml" />
</code>```

Please do help. Tried using 3.0.4 and 3.1.8 cxf versions but thats failing in building the application.

答:
Add this dependency to your project </p>

```<code><dependency>
    <groupId>org.apache.cxf</groupId>
    <artifactId>cxf-rt-transports-http</artifactId>
    <version>2.7.0</version>
</dependency>
</code>```

Body:
We are building a webservices with Java8 and deploying it to Tomcat8. We get following error during start up.

Caused by: org.springframework.beans.factory.BeanDefinitionStoreException: IOException parsing XML document from class path resource [META-INF/cxf/cxf-servlet.xml]; nested exception is java.io.FileNotFoundException: class path resource [META-INF/cxf/cxf-servlet.xml] cannot be opened because it does not exist

When we build the same component with Java7 we are not seeing this issue.

Here is our pom dependency

   <dependency>
        <groupId>org.apache.cxf</groupId>
        <artifactId>cxf-rt-frontend-jaxws</artifactId>
        <version>2.7.0</version>
    </dependency>

Here is our web.xml

<context-param>
    <param-name>contextConfigLocation</param-name>
    <param-value>classpath:applicationConfig.xml</param-value>
</context-param>

<listener>
    <listener-class>
        org.springframework.web.context.ContextLoaderListener
    </listener-class>
</listener>

<servlet>
    <servlet-name>CXFServlet</servlet-name>
    <display-name>CXF Servlet</display-name>
    <servlet-class>
        org.apache.cxf.transport.servlet.CXFServlet
    </servlet-class>
    <load-on-startup>1</load-on-startup>
</servlet>

<servlet-mapping>
    <servlet-name>CXFServlet</servlet-name>
    <url-pattern>/*</url-pattern>
</servlet-mapping>
<welcome-file-list>
    <welcome-file>index.jsp</welcome-file>
</welcome-file-list>

ApplicationConfig.xml has the import

<import resource="classpath:META-INF/cxf/cxf.xml" />
<import resource="classpath:META-INF/cxf/cxf-servlet.xml" />

Please do help. Tried using 3.0.4 and 3.1.8 cxf versions but thats failing in building the application.

Body:
Add this dependency to your project

<dependency>
    <groupId>org.apache.cxf</groupId>
    <artifactId>cxf-rt-transports-http</artifactId>
    <version>2.7.0</version>
</dependency>


问:
Get autocomplete options by AJAX according to typed value
I need to use autocomplete (of jQuery UI) in order to suggest usernames from my database as you type.

The regular way is to use autocomplete's source, but my website has tons of usernames, and I don't want to have one giant string to keep the usernames.
I was thinking about a better way to do it:

When the user types the first letter(s) of a username, an AJAX post request is called to a PHP file, that gets the usernames that start with the letter.
Then, it returns the list to the jQuery, and then process it and make the result the options for the autocomplete.

Is this way better than having one big string with all options? I couldn't code my way myself (couldn't figure out how to turn the mysql result to autocomplete options). And what's the right way to code this? Thanks!

答:
This is an example of what I've done before:

JS

```<code>$("#username").autocomplete({
    source: "autocomplete.php",   // Source for autocomplete must be an array, echoed out via json
    minLength: 1 // Min length before autocomplete starts searching the array from source
  });
</code>```

PHP

```<code>// Pull in searchterm from username, $_GET['TERM'] is a built-in variable for autocomplete
$term = strtolower($_GET['term']);

$searchTerm = "$term%";
$query = "SELECT distinct username from access WHERE username LIKE ? ORDER BY username";
$stmt = $db->prepare($query);
if($stmt){
  $stmt->bind_param("s",$searchTerm);
  $stmt->execute();
  $stmt->bind_result($username);
}

// Create array for autocomplete to search
$usernameList = array();
while ($stmt->fetch()){
        $usernameList[] = $username;  // Assign names to the list
}
// Echo json data for autocomplete to retrieve
echo json_encode($usernameList);
$db->close();
</code>```

HTML

```<code><input type="text" name="username[]" id="username" />
</code>```

Body:
This is an example of what I've done before:

JS

$("#username").autocomplete({
    source: "autocomplete.php",   // Source for autocomplete must be an array, echoed out via json
    minLength: 1 // Min length before autocomplete starts searching the array from source
  });

PHP

// Pull in searchterm from username, $_GET['TERM'] is a built-in variable for autocomplete
$term = strtolower($_GET['term']);

$searchTerm = "$term%";
$query = "SELECT distinct username from access WHERE username LIKE ? ORDER BY username";
$stmt = $db->prepare($query);
if($stmt){
  $stmt->bind_param("s",$searchTerm);
  $stmt->execute();
  $stmt->bind_result($username);
}

// Create array for autocomplete to search
$usernameList = array();
while ($stmt->fetch()){
        $usernameList[] = $username;  // Assign names to the list
}
// Echo json data for autocomplete to retrieve
echo json_encode($usernameList);
$db->close();

HTML

<input type="text" name="username[]" id="username" />


问:
Using Lambda to call Amazon Rekognition on uploaded images
Does anyone know if it is possible to use Lambda to call Rekognition functions when an image is uploaded to an S3 Bucket?

I am looking at integrating a Raspberry Pi device with a Pi Cam to take photos and do some face recognition.

答:
You need to configure an Event on Amazon S3 to call a lambda function that can be either in C#, Java, Python or NodeJS.

For details, see: Tutorial: Creating an Amazon Rekognition Lambda Application


问:
How can I map postgresql json data type using Hibernate?
I am following the example mentioned in the below URL ?
Mapping PostgreSQL JSON column to a Hibernate entity property

But always get the following exception:

```<code>Caused by: org.hibernate.MappingException: No Dialect mapping for JDBC type: 2000
    at org.hibernate.dialect.TypeNames.get(TypeNames.java:76)
    at org.hibernate.dialect.TypeNames.get(TypeNames.java:99)
    at org.hibernate.dialect.Dialect.getTypeName(Dialect.java:310)
    at org.hibernate.mapping.Column.getSqlType(Column.java:226)
    at org.hibernate.mapping.Table.validateColumns(Table.java:369)
    at org.hibernate.cfg.Configuration.validateSchema(Configuration.java:1305)
    at org.hibernate.tool.hbm2ddl.SchemaValidator.validate(SchemaValidator.java:155)
    at org.hibernate.internal.SessionFactoryImpl.<init>(SessionFactoryImpl.java:512)
</code>```

I am using TomEE as the Server. and trying to store Json body to a postgresql column. I am trying to map the entity pojos to the postgres datatype structure.

Any idea what could be the issue ? or does any has a better technique to handle such as scenario ? Please point me to that source.

The script used to create the entity table is:

    CREATE TABLE historyentity
    (
      id character varying(255) NOT NULL,
      userid character varying(255),
      lastchanged timestamp without time zone,
      type character varying(255),
      history json [],
      CONSTRAINT historyentity_pkey PRIMARY KEY (id),
      CONSTRAINT historyentity_userid_fkey FOREIGN KEY (userid)
          REFERENCES userentity (id) MATCH SIMPLE
          ON UPDATE NO ACTION ON DELETE NO ACTION
    )
    WITH (
      OIDS=FALSE
    );
    ALTER TABLE historyentity
      OWNER TO postgres;
    GRANT ALL ON TABLE historyentity TO postgres;
```
<p>Entity Pojos look like as follows:</p>
<pre class="lang-java prettyprint-override"><code>    @Entity
    @Data
    @AllArgsConstructor
    @NoArgsConstructor
    @TypeDefs({ @TypeDef(name = "StringJsonObject", typeClass = StringJsonUserType.class) })
    public class HistoryEntity {

        @Id
        private String id;
        private String userid;
        private String type;
        @Type(type = "StringJsonObject")
        private String history;
        private Date lastchanged;

    }
</code>```

I am using lombok to define the entity pojos.

Following is the Dialect extended class:
I have tried with both the registered types, Column and Hibenate. But both are not working out.

    import org.hibernate.dialect.PostgreSQL82Dialect;

    public class JsonPostgreSQLDialect extends PostgreSQL82Dialect

        {
            @Inject
            public JsonPostgreSQLDialect()
            {
                super();
                   this.registerColumnType(Types.JAVA_OBJECT, "json");
                // this.registerHibernateType(Types.JAVA_OBJECT, "json");
            }
        }
```
<p>The following class is being used to define the User Type:</p>
<pre class="lang-java prettyprint-override"><code>    import java.io.Serializable;
    import java.sql.PreparedStatement;
    import java.sql.ResultSet;
    import java.sql.SQLException;
    import java.sql.Types;

    import org.hibernate.HibernateException;
    import org.hibernate.engine.spi.SessionImplementor;
    import org.hibernate.usertype.UserType;

    public class StringJsonUserType implements UserType
    {
        private final int[] sqlTypesSupported = new int[]{ Types.JAVA_OBJECT };

        /**
         * Return the SQL type codes for the columns mapped by this type. The codes are defined on <tt>java.sql.Types</tt>.
         *
         * @return int[] the typecodes
         * @see java.sql.Types
         */
        @Override
        public int[] sqlTypes()
        {
            return sqlTypesSupported;
        }

        /**
         * The class returned by <tt>nullSafeGet()</tt>.
         *
         * @return Class
         */
        @Override
        public Class returnedClass()
        {
            return String.class;
        }

        /**
         * Compare two instances of the class mapped by this type for persistence "equality". Equality of the persistent
         * state.
         *
         * @return boolean
         */
        @Override
        public boolean equals(Object x, Object y) throws HibernateException
        {

            if (x == null)
            {

                return y == null;
            }

            return x.equals(y);
        }

        /**
         * Get a hashcode for the instance, consistent with persistence "equality"
         */
        @Override
        public int hashCode(Object x) throws HibernateException
        {

            return x.hashCode();
        }

        /**
         * Retrieve an instance of the mapped class from a JDBC resultset. Implementors should handle possibility of null
         * values.
         *
         * @param rs a JDBC result set
         * @param names the column names
         * @param owner the containing entity  @return Object
         */
        @Override
        public Object nullSafeGet(ResultSet rs, String[] names, SessionImplementor session, Object owner)
            throws HibernateException, SQLException
        {
            if (rs.getString(names[0]) == null)
            {
                return null;
            }
            return rs.getString(names[0]);
        }

        /**
         * Write an instance of the mapped class to a prepared statement. Implementors should handle possibility of null
         * values. A multi-column type should be written to parameters starting from <tt>index</tt>.
         *
         * @param st a JDBC prepared statement
         * @param value the object to write
         * @param index statement parameter index
         */
        @Override
        public void nullSafeSet(PreparedStatement st, Object value, int index, SessionImplementor session)
            throws HibernateException, SQLException
        {
            if (value == null)
            {
                st.setNull(index, Types.OTHER);
                return;
            }

            st.setObject(index, value, Types.OTHER);
        }

        /**
         * Return a deep copy of the persistent state, stopping at entities and at collections. It is not necessary to copy
         * immutable objects, or null values, in which case it is safe to simply return the argument.
         *
         * @param value the object to be cloned, which may be null
         * @return Object a copy
         */
        @Override
        public Object deepCopy(Object value) throws HibernateException
        {

            return value;
        }

        /**
         * Are objects of this type mutable?
         *
         * @return boolean
         */
        @Override
        public boolean isMutable()
        {
            return true;
        }

        /**
         * Transform the object into its cacheable representation. At the very least this method should perform a deep copy
         * if the type is mutable. That may not be enough for some implementations, however; for example, associations must
         * be cached as identifier values. (optional operation)
         *
         * @param value the object to be cached
         * @return a cachable representation of the object
         */
        @Override
        public Serializable disassemble(Object value) throws HibernateException
        {
            return (String) this.deepCopy(value);
        }

        /**
         * Reconstruct an object from the cacheable representation. At the very least this method should perform a deep copy
         * if the type is mutable. (optional operation)
         *
         * @param cached the object to be cached
         * @param owner the owner of the cached object
         * @return a reconstructed object from the cachable representation
         */
        @Override
        public Object assemble(Serializable cached, Object owner) throws HibernateException
        {
            return this.deepCopy(cached);
        }

        /**
         * During merge, replace the existing (target) value in the entity we are merging to with a new (original) value
         * from the detached entity we are merging. For immutable objects, or null values, it is safe to simply return the
         * first parameter. For mutable objects, it is safe to return a copy of the first parameter. For objects with
         * component values, it might make sense to recursively replace component values.
         *
         * @param original the value from the detached entity being merged
         * @param target the value in the managed entity
         * @return the value to be merged
         */
        @Override
        public Object replace(Object original, Object target, Object owner) throws HibernateException
        {
            return original;
        }
    }
</code>```

答:
Postgres JSON type has been added to Hibernate in the PostgreSQL92Dialect. So you should either use that dialect or one of its subclasses, or make a custom dialect that adds the following type definition:

```<code>        this.registerColumnType(2000, "json");
</code>```

The type itself can be defined as follows (example for Hibernate 5.x):

```<code>public class JsonType implements UserType {

    public static final ObjectMapper MAPPER = new ObjectMapper();

    private int[] sqlTypes;
    private com.fasterxml.jackson.databind.ObjectWriter writer;
    private JavaType type;
    private boolean isBinary;
    private ObjectReader reader;

    public JsonType() {
        init(SimpleType.constructUnsafe(Object.class), false);
    }

    public JsonType(Class clazz, boolean isBinary) {
        this(SimpleType.construct(clazz), isBinary);
    }

    public JsonType(JavaType type, boolean isBinary) {
        init(type, isBinary);
    }

    protected void init(JavaType type, boolean isBinary) {
        this.type = type;
        this.isBinary = isBinary;
        this.reader = MAPPER.readerFor(type);
        this.writer = MAPPER.writerFor(type);
        this.sqlTypes = new int[]{Types.JAVA_OBJECT};
    }

    public boolean equals(Object x, Object y) throws HibernateException {
        if (x == y) {
            return true;
        } else if (x == null || y == null) {
            return false;
        } else {
            return x.equals(y);
        }
    }

    public int hashCode(Object x) throws HibernateException {
        return null == x ? 0 : x.hashCode();
    }

    public boolean isMutable() {
        return true;
    }

    @Override
    public Object nullSafeGet(ResultSet rs, String[] names, SharedSessionContractImplementor session, Object owner) throws HibernateException, SQLException {
        final Object result = rs.getObject(names[0]);
        if (!rs.wasNull()) {
            String content;

            if (result instanceof String) {
                content = (String) result;
            } else if (result instanceof PGobject) {
                // If we get directly the PGobject for some reason (more exactly, if a DB like H2 does the serialization directly)
                content = ((PGobject) result).getValue();
            } else {
                throw new IllegalArgumentException("Unknown object type (excepted pgobject or json string)");
            }
            if (content != null) {
                return convertJsonToObject(content);
            }
        }
        return null;
    }

    @Override
    public void nullSafeSet(PreparedStatement st, Object value, int index, SharedSessionContractImplementor session) throws HibernateException, SQLException {
        if (value == null) {
            st.setObject(index, null);
            return;
        }
        PGobject pg = new PGobject();
        pg.setType(isBinary ? "jsonb" : "json");
        pg.setValue(convertObjectToJson(value));
        st.setObject(index, pg);
    }

    Object convertJsonToObject(String content) {
        try {
            return reader.readValue(content);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    String convertObjectToJson(Object object) {
        try {
            return writer.writeValueAsString(object);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    public Object deepCopy(Object value) throws HibernateException {
        String json = convertObjectToJson(value);
        return convertJsonToObject(json);
    }

    public Object replace(Object original, Object target, Object owner)
        throws HibernateException {
        return deepCopy(original);
    }

    public Serializable disassemble(Object value) throws HibernateException {
        return (Serializable) deepCopy(value);
    }

    public Object assemble(Serializable cached, Object owner)
        throws HibernateException {
        return deepCopy(cached);
    }

    public int[] sqlTypes() {
        return sqlTypes;
    }

    public Class returnedClass() {
        return type.getRawClass();
    }
}
</code>```

This example uses Jackson as a framework for JSON (de)serialization.

You can then use your type as follows:

```<code>@Entity
@TypeDefs({@TypeDef( name= "StringJsonObject", typeClass = JsonType.class)})
public class MyEntity {

    @Type(type = "StringJsonObject")
    @Column(name="visuals", columnDefinition = "json")
    private Map<String, String> visuals;

}
</code>```

But this is all very similar to the type that you implemented (presumably for Hibernate 4.x). So why wasn't your implementation working? This is because your field is actually of type json[] (a Postgres array of JSON objects). This mapper only works with JSON objects (type json). This JSON object  can very well be a JSON array of JSON objects, but it has to be of type json. So you should change the type in your database schema, or implement a UserType that can work with arrays, but the first option is most likely.

ContentLicense: CC BY-SA 4.0

Body:
I am following the example mentioned in the below URL ?
Mapping PostgreSQL JSON column to a Hibernate entity property

But always get the following exception:

Caused by: org.hibernate.MappingException: No Dialect mapping for JDBC type: 2000
    at org.hibernate.dialect.TypeNames.get(TypeNames.java:76)
    at org.hibernate.dialect.TypeNames.get(TypeNames.java:99)
    at org.hibernate.dialect.Dialect.getTypeName(Dialect.java:310)
    at org.hibernate.mapping.Column.getSqlType(Column.java:226)
    at org.hibernate.mapping.Table.validateColumns(Table.java:369)
    at org.hibernate.cfg.Configuration.validateSchema(Configuration.java:1305)
    at org.hibernate.tool.hbm2ddl.SchemaValidator.validate(SchemaValidator.java:155)
    at org.hibernate.internal.SessionFactoryImpl.<init>(SessionFactoryImpl.java:512)

I am using TomEE as the Server. and trying to store Json body to a postgresql column. I am trying to map the entity pojos to the postgres datatype structure.

Any idea what could be the issue ? or does any has a better technique to handle such as scenario ? Please point me to that source.

The script used to create the entity table is:

    CREATE TABLE historyentity
    (
      id character varying(255) NOT NULL,
      userid character varying(255),
      lastchanged timestamp without time zone,
      type character varying(255),
      history json [],
      CONSTRAINT historyentity_pkey PRIMARY KEY (id),
      CONSTRAINT historyentity_userid_fkey FOREIGN KEY (userid)
          REFERENCES userentity (id) MATCH SIMPLE
          ON UPDATE NO ACTION ON DELETE NO ACTION
    )
    WITH (
      OIDS=FALSE
    );
    ALTER TABLE historyentity
      OWNER TO postgres;
    GRANT ALL ON TABLE historyentity TO postgres;

Entity Pojos look like as follows:

    @Entity
    @Data
    @AllArgsConstructor
    @NoArgsConstructor
    @TypeDefs({ @TypeDef(name = "StringJsonObject", typeClass = StringJsonUserType.class) })
    public class HistoryEntity {

        @Id
        private String id;
        private String userid;
        private String type;
        @Type(type = "StringJsonObject")
        private String history;
        private Date lastchanged;

    }

I am using lombok to define the entity pojos.

Following is the Dialect extended class:
I have tried with both the registered types, Column and Hibenate. But both are not working out.

    import org.hibernate.dialect.PostgreSQL82Dialect;

    public class JsonPostgreSQLDialect extends PostgreSQL82Dialect

        {
            @Inject
            public JsonPostgreSQLDialect()
            {
                super();
                   this.registerColumnType(Types.JAVA_OBJECT, "json");
                // this.registerHibernateType(Types.JAVA_OBJECT, "json");
            }
        }

The following class is being used to define the User Type:

    import java.io.Serializable;
    import java.sql.PreparedStatement;
    import java.sql.ResultSet;
    import java.sql.SQLException;
    import java.sql.Types;

    import org.hibernate.HibernateException;
    import org.hibernate.engine.spi.SessionImplementor;
    import org.hibernate.usertype.UserType;

    public class StringJsonUserType implements UserType
    {
        private final int[] sqlTypesSupported = new int[]{ Types.JAVA_OBJECT };

        /**
         * Return the SQL type codes for the columns mapped by this type. The codes are defined on <tt>java.sql.Types</tt>.
         *
         * @return int[] the typecodes
         * @see java.sql.Types
         */
        @Override
        public int[] sqlTypes()
        {
            return sqlTypesSupported;
        }

        /**
         * The class returned by <tt>nullSafeGet()</tt>.
         *
         * @return Class
         */
        @Override
        public Class returnedClass()
        {
            return String.class;
        }

        /**
         * Compare two instances of the class mapped by this type for persistence "equality". Equality of the persistent
         * state.
         *
         * @return boolean
         */
        @Override
        public boolean equals(Object x, Object y) throws HibernateException
        {

            if (x == null)
            {

                return y == null;
            }

            return x.equals(y);
        }

        /**
         * Get a hashcode for the instance, consistent with persistence "equality"
         */
        @Override
        public int hashCode(Object x) throws HibernateException
        {

            return x.hashCode();
        }

        /**
         * Retrieve an instance of the mapped class from a JDBC resultset. Implementors should handle possibility of null
         * values.
         *
         * @param rs a JDBC result set
         * @param names the column names
         * @param owner the containing entity  @return Object
         */
        @Override
        public Object nullSafeGet(ResultSet rs, String[] names, SessionImplementor session, Object owner)
            throws HibernateException, SQLException
        {
            if (rs.getString(names[0]) == null)
            {
                return null;
            }
            return rs.getString(names[0]);
        }

        /**
         * Write an instance of the mapped class to a prepared statement. Implementors should handle possibility of null
         * values. A multi-column type should be written to parameters starting from <tt>index</tt>.
         *
         * @param st a JDBC prepared statement
         * @param value the object to write
         * @param index statement parameter index
         */
        @Override
        public void nullSafeSet(PreparedStatement st, Object value, int index, SessionImplementor session)
            throws HibernateException, SQLException
        {
            if (value == null)
            {
                st.setNull(index, Types.OTHER);
                return;
            }

            st.setObject(index, value, Types.OTHER);
        }

        /**
         * Return a deep copy of the persistent state, stopping at entities and at collections. It is not necessary to copy
         * immutable objects, or null values, in which case it is safe to simply return the argument.
         *
         * @param value the object to be cloned, which may be null
         * @return Object a copy
         */
        @Override
        public Object deepCopy(Object value) throws HibernateException
        {

            return value;
        }

        /**
         * Are objects of this type mutable?
         *
         * @return boolean
         */
        @Override
        public boolean isMutable()
        {
            return true;
        }

        /**
         * Transform the object into its cacheable representation. At the very least this method should perform a deep copy
         * if the type is mutable. That may not be enough for some implementations, however; for example, associations must
         * be cached as identifier values. (optional operation)
         *
         * @param value the object to be cached
         * @return a cachable representation of the object
         */
        @Override
        public Serializable disassemble(Object value) throws HibernateException
        {
            return (String) this.deepCopy(value);
        }

        /**
         * Reconstruct an object from the cacheable representation. At the very least this method should perform a deep copy
         * if the type is mutable. (optional operation)
         *
         * @param cached the object to be cached
         * @param owner the owner of the cached object
         * @return a reconstructed object from the cachable representation
         */
        @Override
        public Object assemble(Serializable cached, Object owner) throws HibernateException
        {
            return this.deepCopy(cached);
        }

        /**
         * During merge, replace the existing (target) value in the entity we are merging to with a new (original) value
         * from the detached entity we are merging. For immutable objects, or null values, it is safe to simply return the
         * first parameter. For mutable objects, it is safe to return a copy of the first parameter. For objects with
         * component values, it might make sense to recursively replace component values.
         *
         * @param original the value from the detached entity being merged
         * @param target the value in the managed entity
         * @return the value to be merged
         */
        @Override
        public Object replace(Object original, Object target, Object owner) throws HibernateException
        {
            return original;
        }
    }

Body:
Postgres JSON type has been added to Hibernate in the PostgreSQL92Dialect. So you should either use that dialect or one of its subclasses, or make a custom dialect that adds the following type definition:

        this.registerColumnType(2000, "json");

The type itself can be defined as follows (example for Hibernate 5.x):

public class JsonType implements UserType {

    public static final ObjectMapper MAPPER = new ObjectMapper();

    private int[] sqlTypes;
    private com.fasterxml.jackson.databind.ObjectWriter writer;
    private JavaType type;
    private boolean isBinary;
    private ObjectReader reader;

    public JsonType() {
        init(SimpleType.constructUnsafe(Object.class), false);
    }

    public JsonType(Class clazz, boolean isBinary) {
        this(SimpleType.construct(clazz), isBinary);
    }

    public JsonType(JavaType type, boolean isBinary) {
        init(type, isBinary);
    }

    protected void init(JavaType type, boolean isBinary) {
        this.type = type;
        this.isBinary = isBinary;
        this.reader = MAPPER.readerFor(type);
        this.writer = MAPPER.writerFor(type);
        this.sqlTypes = new int[]{Types.JAVA_OBJECT};
    }

    public boolean equals(Object x, Object y) throws HibernateException {
        if (x == y) {
            return true;
        } else if (x == null || y == null) {
            return false;
        } else {
            return x.equals(y);
        }
    }

    public int hashCode(Object x) throws HibernateException {
        return null == x ? 0 : x.hashCode();
    }

    public boolean isMutable() {
        return true;
    }

    @Override
    public Object nullSafeGet(ResultSet rs, String[] names, SharedSessionContractImplementor session, Object owner) throws HibernateException, SQLException {
        final Object result = rs.getObject(names[0]);
        if (!rs.wasNull()) {
            String content;

            if (result instanceof String) {
                content = (String) result;
            } else if (result instanceof PGobject) {
                // If we get directly the PGobject for some reason (more exactly, if a DB like H2 does the serialization directly)
                content = ((PGobject) result).getValue();
            } else {
                throw new IllegalArgumentException("Unknown object type (excepted pgobject or json string)");
            }
            if (content != null) {
                return convertJsonToObject(content);
            }
        }
        return null;
    }

    @Override
    public void nullSafeSet(PreparedStatement st, Object value, int index, SharedSessionContractImplementor session) throws HibernateException, SQLException {
        if (value == null) {
            st.setObject(index, null);
            return;
        }
        PGobject pg = new PGobject();
        pg.setType(isBinary ? "jsonb" : "json");
        pg.setValue(convertObjectToJson(value));
        st.setObject(index, pg);
    }

    Object convertJsonToObject(String content) {
        try {
            return reader.readValue(content);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    String convertObjectToJson(Object object) {
        try {
            return writer.writeValueAsString(object);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    public Object deepCopy(Object value) throws HibernateException {
        String json = convertObjectToJson(value);
        return convertJsonToObject(json);
    }

    public Object replace(Object original, Object target, Object owner)
        throws HibernateException {
        return deepCopy(original);
    }

    public Serializable disassemble(Object value) throws HibernateException {
        return (Serializable) deepCopy(value);
    }

    public Object assemble(Serializable cached, Object owner)
        throws HibernateException {
        return deepCopy(cached);
    }

    public int[] sqlTypes() {
        return sqlTypes;
    }

    public Class returnedClass() {
        return type.getRawClass();
    }
}

This example uses Jackson as a framework for JSON (de)serialization.

You can then use your type as follows:

@Entity
@TypeDefs({@TypeDef( name= "StringJsonObject", typeClass = JsonType.class)})
public class MyEntity {

    @Type(type = "StringJsonObject")
    @Column(name="visuals", columnDefinition = "json")
    private Map<String, String> visuals;

}

But this is all very similar to the type that you implemented (presumably for Hibernate 4.x). So why wasn't your implementation working? This is because your field is actually of type json[] (a Postgres array of JSON objects). This mapper only works with JSON objects (type json). This JSON object  can very well be a JSON array of JSON objects, but it has to be of type json. So you should change the type in your database schema, or implement a UserType that can work with arrays, but the first option is most likely.


问:
Attempt to assign property of non-object in Laravel 5
I have the following Laravel code which receive data from an API and store them on a database.

```<code>$sample = Sample::findOrFail($id);
$new_sample = $request->except('filename');

if ( $request->file('filename') ) {
     if ( $sample->filename ) {
           Storage::disk('documents')->delete($sample->filename);
     }
     $file = $request->file('filename');
     $extension = $file->getClientOriginalExtension();
     $new_sample->filename = $file->getFilename().'.'.$extension;
     Storage::disk('documents')->put($file->getFilename().'.'.$extension,  File::get($file));
}

if (!$sample->update($new_sample)) {
     throw new HttpException(500);
}
</code>```

I have the following error Attempt to assign property of non-object and I found that by commenting out the line where I assign the filename, the error doesn't appear again.

$new_sample->filename = $file->getFilename().'.'.$extension;

So, I guess the problem is here. I tried also with

$new_sample['filename'] = $file->getFilename().'.'.$extension;

but then it doesn't store the filename.

Any idea?

答:
You get this error because there is no file in Request object.

This is because you're using the wrong method for checking file. Use hasFile() method. So you need to change this:

```<code>if ( $request->file('filename') ) {
</code>```

To this:

```<code>if ($request->hasFile('filename')) {
</code>```

Body:
I have the following Laravel code which receive data from an API and store them on a database.

$sample = Sample::findOrFail($id);
$new_sample = $request->except('filename');

if ( $request->file('filename') ) {
     if ( $sample->filename ) {
           Storage::disk('documents')->delete($sample->filename);
     }
     $file = $request->file('filename');
     $extension = $file->getClientOriginalExtension();
     $new_sample->filename = $file->getFilename().'.'.$extension;
     Storage::disk('documents')->put($file->getFilename().'.'.$extension,  File::get($file));
}

if (!$sample->update($new_sample)) {
     throw new HttpException(500);
}

I have the following error Attempt to assign property of non-object and I found that by commenting out the line where I assign the filename, the error doesn't appear again.

$new_sample->filename = $file->getFilename().'.'.$extension;

So, I guess the problem is here. I tried also with

$new_sample['filename'] = $file->getFilename().'.'.$extension;

but then it doesn't store the filename.

Any idea?

Body:
You get this error because there is no file in Request object.

This is because you're using the wrong method for checking file. Use hasFile() method. So you need to change this:

if ( $request->file('filename') ) {

To this:

if ($request->hasFile('filename')) {


问:
C# - Multiple Possibilities for Property Types
I have an Interface called "IOperation"

```<code>interface IOperation<T,U>
{
    T FirstOperand { get; set; }
    U SecondOperand { get; set; }
    int Result { get; }
}
</code>```

FirstOperand and SecondOperand can either be an int or another IOperation. What is the best way to implement such behaviour? Example of an inherited class:

```<code>class Subtraction<T, U> : IOperation<T, U>
{
    public T FirstOperand { get; set; }
    public U SecondOperand { get; set; }

    public int Result
    {
        get
        {
            var first = 0;
            var second = 0;
            if(FirstOperand is Double)
            {
                first = (Double) FirstOperand;
            }
            if(FirstOperand is IOperation)
            {
                first = (IOperation) FirstOperand;
            }
            // TODO: Same thing with SecondOperand

            return FirstOperand - SecondOperand;
        }
    }
}
</code>```

Well, compilation error, apparently a direct cast doesn't work for generics. This problem could be fixed by casting into object first, but I'm pretty sure that would be a terrible approach and not good practice at all. Do you have any suggestions how I can solve this problem in a good way?

答:
There is a way you could do it (I don't recommend it though).

You could pack both of your operands in a dynamic object. This will compile:

```<code> class Subtraction<T, U> : IOperation<T, U>
 {
     public T FirstOperand { get; set; }
     public U SecondOperand { get; set; }

     public int Result
     {
         get
         {
             dynamic first = FirstOperand;
             dynamic second = SecondOperand;
             return (int)(first - second);
         }
     }
 }
</code>```

And it will work for types that have an operator - overload for the according types (i.e. int and double) and that have an implicit or explicit cast to int.

However, using dynamic objects has two major problems:

type safety: this code will compile regardless of the actual types of T and U. If the types are not compatible (they don't have a common operator), you will get an exception at runtime.

performance: dynamic objetcs build expression trees that are compiled on first use and than cached at runtime. This means, that your program will have to compile things at runtime which ends in slowing down your code (however, only once for each type combination that is used).

For additional information read the article about dynamic and
expression trees

Body:
I have an Interface called "IOperation"

interface IOperation<T,U>
{
    T FirstOperand { get; set; }
    U SecondOperand { get; set; }
    int Result { get; }
}

FirstOperand and SecondOperand can either be an int or another IOperation. What is the best way to implement such behaviour? Example of an inherited class:

class Subtraction<T, U> : IOperation<T, U>
{
    public T FirstOperand { get; set; }
    public U SecondOperand { get; set; }

    public int Result
    {
        get
        {
            var first = 0;
            var second = 0;
            if(FirstOperand is Double)
            {
                first = (Double) FirstOperand;
            }
            if(FirstOperand is IOperation)
            {
                first = (IOperation) FirstOperand;
            }
            // TODO: Same thing with SecondOperand

            return FirstOperand - SecondOperand;
        }
    }
}

Well, compilation error, apparently a direct cast doesn't work for generics. This problem could be fixed by casting into object first, but I'm pretty sure that would be a terrible approach and not good practice at all. Do you have any suggestions how I can solve this problem in a good way?

Body:
There is a way you could do it (I don't recommend it though).

You could pack both of your operands in a dynamic object. This will compile:

 class Subtraction<T, U> : IOperation<T, U>
 {
     public T FirstOperand { get; set; }
     public U SecondOperand { get; set; }

     public int Result
     {
         get
         {
             dynamic first = FirstOperand;
             dynamic second = SecondOperand;
             return (int)(first - second);
         }
     }
 }

And it will work for types that have an operator - overload for the according types (i.e. int and double) and that have an implicit or explicit cast to int.

However, using dynamic objects has two major problems:

type safety: this code will compile regardless of the actual types of T and U. If the types are not compatible (they don't have a common operator), you will get an exception at runtime.

performance: dynamic objetcs build expression trees that are compiled on first use and than cached at runtime. This means, that your program will have to compile things at runtime which ends in slowing down your code (however, only once for each type combination that is used).

For additional information read the article about dynamic and
expression trees


问:
Create job from Dataflow template from Go app
I'm trying to launch a Dataflow job from an existing template, via a Go app.

So far, I've brought in google.golang.org/api/dataflow/v1b3 and created a CreateJobFromTemplateRequest with the job info.

How can I now execute that request using the built-in service account credentials in Compute Engine?

答:
Using Auto-generated Google APIs for Go is only recommended if there is a Google Client Library for Go developed for the service you are calling. There is not a client library yet for Dataflow.

To launch a Dataflow template from a Go app using default credentials:

```<code>ctx := context.Background()
oauthClient, err := google.DefaultClient(ctx, dataflow.CloudPlatformScope)

dataflowService, err := dataflow.New(oauthClient)

if err != nil {
  panic(err)
}

templateRequest := dataflow.CreateJobFromTemplateRequest{
  GcsPath: "gcs path to template here",
  JobName: "choose a unique job name here",
  Parameters: map[string]string{
    "parameters": "for job",
  },
}

result, err := dataflowService.Projects.Templates.Create("project id", &templateRequest).Do()

if err != nil {
  panic(err)
}
</code>```

Body:
Using Auto-generated Google APIs for Go is only recommended if there is a Google Client Library for Go developed for the service you are calling. There is not a client library yet for Dataflow.

To launch a Dataflow template from a Go app using default credentials:

ctx := context.Background()
oauthClient, err := google.DefaultClient(ctx, dataflow.CloudPlatformScope)

dataflowService, err := dataflow.New(oauthClient)

if err != nil {
  panic(err)
}

templateRequest := dataflow.CreateJobFromTemplateRequest{
  GcsPath: "gcs path to template here",
  JobName: "choose a unique job name here",
  Parameters: map[string]string{
    "parameters": "for job",
  },
}

result, err := dataflowService.Projects.Templates.Create("project id", &templateRequest).Do()

if err != nil {
  panic(err)
}


问:
How often does puppet with splay run?
Say I turn on the splay option for a puppet agent with:

```<code>sudo puppet config set splay true --section agent
</code>```

Then I run:

```<code>/usr/local/bin/puppet agent --onetime --no-daemonize
</code>```

From what I understand splay causes the check-in time to randomised.

I'm wondering how this randomness is calculated? Is it a random time in the next, hour, day, week, or something else?

答:
I'm reading here, that the default time is randomised between the time the command is run and 2 minutes after that.

When the service is stopped, the selected nodes will randomly stagger
  the start of their runs (“splay”) over a default interval of two
  minutes. If you wish, you can specify special options, including a
  longer interval (“splaylimit”). You can also set the force option to
  true if you want the selected nodes to start immediately. This
  behavior is more flexible and resilient.

The 2 minutes can be modified with:

```<code>LIMIT=60
sudo puppet config set splaylimit $LIMIT --section agent
</code>```

Title: How often does puppet with splay run?

Body:
Say I turn on the splay option for a puppet agent with:

sudo puppet config set splay true --section agent

Then I run:

/usr/local/bin/puppet agent --onetime --no-daemonize

From what I understand splay causes the check-in time to randomised.

I'm wondering how this randomness is calculated? Is it a random time in the next, hour, day, week, or something else?

Body:
I'm reading here, that the default time is randomised between the time the command is run and 2 minutes after that.

When the service is stopped, the selected nodes will randomly stagger
  the start of their runs (“splay”) over a default interval of two
  minutes. If you wish, you can specify special options, including a
  longer interval (“splaylimit”). You can also set the force option to
  true if you want the selected nodes to start immediately. This
  behavior is more flexible and resilient.

The 2 minutes can be modified with:

LIMIT=60
sudo puppet config set splaylimit $LIMIT --section agent


问:
Dropdown list selecting a filter
I'm trying to get every properties of a json object in order to create a dropdown list.

I'm going to explain myself, for example :

I've got 2 json objects :

```<code>{"name":"Paul","age":"18","sport":"Basket","color":"Green"}
{"name":"Jhon","age":"20","sport":"Basket","fruit":"banana", "number":"5"}
</code>```

I've got a dropdown list within {name,age,sport,color,fruit,number}.
If I select name, I've got a second dropdown list within {Paul,Jhon}.

I'm using AngularJS, Node.js

I know that after getting every properties I have to apply unicity filter, to only have the differents properties but not the occurence

答:
May not be perfect but it is working

HTML:

```<code> <div ng-controller="MyCtrl">
      <select ng-model="selectedItem" ng-change="itemChanged()" ng-options="item for item in list1">
      </select>
      <select ng-model="second" ng-options="lst for lst in list2 "></select>

 </div>
</code>```

Controller:

```<code>var myApp = angular.module('myApp', []);

myApp.controller('MyCtrl', function MyCtrl($scope) {
$scope.list1 = ["name","age","sport","color","fruit","number"];
$scope.data = [{"name":"Paul","age":"18","sport":"Basket","color":"Green"},{"name":"Jhon","age":"20","sport":"Basket","fruit":"banana", "number":"5"}];
  $scope.itemChanged = function() {
   var prop = $scope.selectedItem;
   $scope.list2 = [];
   $scope.data.forEach(x => {
     if(x[prop] && $scope.list2.indexOf(x[prop])) {
       $scope.list2.push(x[prop]);
     }
   });
  };
});
</code>```

Jsfiddle Demo: http://jsfiddle.net/sathvike/vzzmrnvL/

Title: Dropdown list selecting a filter

Body:
I'm trying to get every properties of a json object in order to create a dropdown list.

I'm going to explain myself, for example :

I've got 2 json objects :

{"name":"Paul","age":"18","sport":"Basket","color":"Green"}
{"name":"Jhon","age":"20","sport":"Basket","fruit":"banana", "number":"5"}

I've got a dropdown list within {name,age,sport,color,fruit,number}.
If I select name, I've got a second dropdown list within {Paul,Jhon}.

I'm using AngularJS, Node.js

I know that after getting every properties I have to apply unicity filter, to only have the differents properties but not the occurence

Body:
May not be perfect but it is working

HTML:

 <div ng-controller="MyCtrl">
      <select ng-model="selectedItem" ng-change="itemChanged()" ng-options="item for item in list1">
      </select>
      <select ng-model="second" ng-options="lst for lst in list2 "></select>

 </div>

Controller:

var myApp = angular.module('myApp', []);

myApp.controller('MyCtrl', function MyCtrl($scope) {
$scope.list1 = ["name","age","sport","color","fruit","number"];
$scope.data = [{"name":"Paul","age":"18","sport":"Basket","color":"Green"},{"name":"Jhon","age":"20","sport":"Basket","fruit":"banana", "number":"5"}];
  $scope.itemChanged = function() {
   var prop = $scope.selectedItem;
   $scope.list2 = [];
   $scope.data.forEach(x => {
     if(x[prop] && $scope.list2.indexOf(x[prop])) {
       $scope.list2.push(x[prop]);
     }
   });
  };
});

Jsfiddle Demo: http://jsfiddle.net/sathvike/vzzmrnvL/


问:
Resize Table Range
This section of code is intended to resize a table in order to add a new row of data input from a UserForm. Currently when I run the code, I get:

Error 9 - Subscript out of range

On the line:

```<code>Set tbl1 = sht1.ListObjects("LogTable")
</code>```

This is the full code:

```<code>Private Sub CommandButton1_Click()

'Start button
'Log start time as the time the button is clicked

Dim EmptyRow, LastRow   As Long
Dim sPath               As String
Dim tbl1, tbl2          As Range

'Find next empty row based on first column
Set wb = ThisWorkbook
Set sht1 = wb.Sheets("Log")
Set tbl1 = sht1.ListObjects("LogTable")

LastRow = sht1.Cells(sht1.Rows.Count, "A").End(xlUp).Row
EmptyRow = LastRow + 1

tbl1.Resize Range("A2:E" & EmptyRow)

sht1.Cells(EmptyRow, 1).Value = Now
</code>```

Thanks for your help!

答:
Try this code:

```<code>Dim tbl1 As ListObject
Dim wb As Workbook
Dim sht1 As Worksheet

    Set wb = ThisWorkbook
    Set sht1 = wb.Sheets("Log")
    Set tbl1 = sht1.ListObjects("LogTable")

    tbl1.ListRows.Add
</code>```

Please note:

Your code contains incorrect declarations: EmptyRow and tbl1 are declared as Variant, which is not what you intended (it seems). It's a common VBA mistake – assuming that type declaration applies to all variables on the line. It does not. The easiest way to avoid it is to always declare one variable per line.

Also, tbl1 and tbl2 should be declared as tables (ListObject), not ranges.

Title: Resize Table Range

Body:
This section of code is intended to resize a table in order to add a new row of data input from a UserForm. Currently when I run the code, I get:

Error 9 - Subscript out of range

On the line:

Set tbl1 = sht1.ListObjects("LogTable")

This is the full code:

Private Sub CommandButton1_Click()

'Start button
'Log start time as the time the button is clicked

Dim EmptyRow, LastRow   As Long
Dim sPath               As String
Dim tbl1, tbl2          As Range

'Find next empty row based on first column
Set wb = ThisWorkbook
Set sht1 = wb.Sheets("Log")
Set tbl1 = sht1.ListObjects("LogTable")

LastRow = sht1.Cells(sht1.Rows.Count, "A").End(xlUp).Row
EmptyRow = LastRow + 1

tbl1.Resize Range("A2:E" & EmptyRow)

sht1.Cells(EmptyRow, 1).Value = Now

Thanks for your help!

Body:
Try this code:

Dim tbl1 As ListObject
Dim wb As Workbook
Dim sht1 As Worksheet

    Set wb = ThisWorkbook
    Set sht1 = wb.Sheets("Log")
    Set tbl1 = sht1.ListObjects("LogTable")

    tbl1.ListRows.Add

Please note:

Your code contains incorrect declarations: EmptyRow and tbl1 are declared as Variant, which is not what you intended (it seems). It's a common VBA mistake – assuming that type declaration applies to all variables on the line. It does not. The easiest way to avoid it is to always declare one variable per line.

Also, tbl1 and tbl2 should be declared as tables (ListObject), not ranges.


问:
How can I call distinct on entities in ASP.NET MVC 4
I have two tables where users save data for cities. I want to count the total number of cities in both tables using a Distinct on the join query.

```<code>Table 1       Table 2

ID City       ID City
--------      ----------
1  CHD        1  PUT
2  PUT        2  CHD
3  LUD        3  MALK
4  CHD        4  PUT
5  PUT        5  APC
</code>```

I want to join both tables and get the total count of Cities, but I want to make sure that cites will not be repeated in the count. That is, if CHD exists in table 1 and Table 2 then only count it once.

I don't want to count cities repeatedly. I want to be able to filter my results and get the total count.

I have written this code for Table 1 but I want the same result with a joined query.

```<code>public int Locations()
{
    int Locations = 0;

    using (var db = new Entities())
        Locations = db.Table1.Select(o => o.City).Distinct().Count();

    int Locations = 0;

    using (var db = new Entities())
        Locations = db.Table2.Select(o => o.City).Distinct().Count();

    return Locations;
}
</code>```

How can I do this with 2 tables?

答:
This is untested, but I think it might be what you're looking for.    </p>

```<code>public int Locations()
{
     using (var db = new Entities())
     {
          return (db.Table1.Select(x => x.City).Union(db.Table2.Select(y => y.City)))
                 .Distinct().Count()
     }
}
</code>```

Body:
I have two tables where users save data for cities. I want to count the total number of cities in both tables using a Distinct on the join query.

Table 1       Table 2

ID City       ID City
--------      ----------
1  CHD        1  PUT
2  PUT        2  CHD
3  LUD        3  MALK
4  CHD        4  PUT
5  PUT        5  APC

I want to join both tables and get the total count of Cities, but I want to make sure that cites will not be repeated in the count. That is, if CHD exists in table 1 and Table 2 then only count it once.

I don't want to count cities repeatedly. I want to be able to filter my results and get the total count.

I have written this code for Table 1 but I want the same result with a joined query.

public int Locations()
{
    int Locations = 0;

    using (var db = new Entities())
        Locations = db.Table1.Select(o => o.City).Distinct().Count();

    int Locations = 0;

    using (var db = new Entities())
        Locations = db.Table2.Select(o => o.City).Distinct().Count();

    return Locations;
}

How can I do this with 2 tables?

Body:
This is untested, but I think it might be what you're looking for.

public int Locations()
{
     using (var db = new Entities())
     {
          return (db.Table1.Select(x => x.City).Union(db.Table2.Select(y => y.City)))
                 .Distinct().Count()
     }
}


问:
Align HTML items
I have one div containing 3 divs.

original

HTML code

.state {
  background-color: rgba(233, 234, 237, 0.9);
  height: 7vh;
  width: 80%;
  border-radius: 14px;
  margin: 10px 0 15px 80px;
  display: flex;
  align-items: stretch;
}

.state-main {
  text-align: center;
  padding-top: 10px;
  font-weight: 900;
  font-size: 14px;
}

.options {
  text-align: right;
  margin-bottom: 10px;
}

.owner-image {
  border-top-left-radius: 14px;
  border-bottom-left-radius: 14px;
}```
<pre class="snippet-code-html lang-html prettyprint-override"><code><div class="state">

  <div class="owner">
    <img class="owner-image" src="img/uk.jpg">
  </div>
  <div class="state-main">
    <p class="state-name">PENNSYLVANIA</p>
  </div>
  <div class="options">
    <p id="time"></p>
    <button>SEND TROOPS</button>
    <button>ATTACK</button>
  </div>

</div></code>```

答:
Use flexbox (browser support).

.state {
  display: flex;
  align-items: center;
  justify-content: space-between;
  min-height: 80px;
  background-color: lightgray;
}

.state,
.btns button {
  text-transform: uppercase;
}```
<pre class="snippet-code-html lang-html prettyprint-override"><code><div class="state">
  <img src="http://placehold.it/150x80/fc0">
  <p>
    Pennsylvania
  </p>
  <div class="btns">
    <button>Send Troops</button>
    <button>Attack</button>
  </div>
</div></code>```

**For IE9 and older you'll need to provide a fallback. Whether or not you need to do this depends on target audience.

Title: Align HTML items

Body:
I have one div containing 3 divs.

original

HTML code

.state {
  background-color: rgba(233, 234, 237, 0.9);
  height: 7vh;
  width: 80%;
  border-radius: 14px;
  margin: 10px 0 15px 80px;
  display: flex;
  align-items: stretch;
}

.state-main {
  text-align: center;
  padding-top: 10px;
  font-weight: 900;
  font-size: 14px;
}

.options {
  text-align: right;
  margin-bottom: 10px;
}

.owner-image {
  border-top-left-radius: 14px;
  border-bottom-left-radius: 14px;
}

PENNSYLVANIA

    <button>SEND TROOPS</button>
    <button>ATTACK</button>

Body:
Use flexbox (browser support).

.state {
  display: flex;
  align-items: center;
  justify-content: space-between;
  min-height: 80px;
  background-color: lightgray;
}

.state,
.btns button {
  text-transform: uppercase;
}

<div class="state">
  <img src="http://placehold.it/150x80/fc0">
  <p>
    Pennsylvania
  </p>
  <div class="btns">
    <button>Send Troops</button>
    <button>Attack</button>
  </div>
</div>

**For IE9 and older you'll need to provide a fallback. Whether or not you need to do this depends on target audience.


问:
c# Paradox: Converting List to Array is Iterating more efficient?
List Benchmark: Size=1000, Runs=50000, Total Time = 19.5 sec

```<code>var list = new List<string>(Size);
for (int i = 0; i < Size; i++) list.Add(i.ToString());

var b = new Benchmark();
b.Test("TestListIteration", () =>
{
   c = 0;
   for (int i = 0; i < Runs; i++)
   {
      for (int j = 0; j < Size; j++)
      {
          c += list[j].Length;
      }
   }
});
</code>```

List To Array Benchmark: Size=1000, Runs=50000, Total Time=15.449

```<code>var list = new List<string>(Size);
for (int i = 0; i < Size; i++) list.Add(i.ToString());

var b = new Benchmark();
b.Test("TestListIteration", () =>
{
   c = 0;
   for (int i = 0; i < Runs; i++)
   {
      var array = list.ToArray(); //Changed line here !!!
      for (int j = 0; j < Size; j++)
      {
          c += array[j].Length;
      }
   }
});
</code>```

Isn't this a paradox ?

How can performing two actions

  a) converting entire list to array and

  b) iterating entire array.

Be faster than doing b alone (iterating list).

If this is the case, then it means than all code written in the world is wrong. We should have predicted for this case. And each "For" loop on a list should automatically call .ToArray before it starts. Even if the array is discarded later.

Edit: Here are the Results depending on the "Size".

Size=10, Runs=5000000: List Wins
List : 20.362, ListToArray: 37.36

Size=100, Runs=500000: List Wins
List : 19.64, ListToArray: 23.162

Size=1000, Runs=50000: ListToArray Wins
List : 19.5, ListToArray: 15.449

Size=10000, Runs=5000: ListToArray Wins
List : 20.094, ListToArray: 14.453

Size=10000000, Runs=5: Computer died

答:
Isn't this a paradox?

No.

How can performing two actions
  a) converting entire list to array and
  b) iterating entire array.
  Be faster than doing b alone (iterating list)?

Converting a list to an array is extremely fast.

Iterating elements of a list is very slightly slower than iterating elements of an array.

Why are these things true?

Because (1) lists are secretly arrays behind the scenes, (2) array-to-array copy is heavily optimized -- it is done by going directly to hardware, not by iterating every element and copying it one at a time -- so list-to-array copying is also heavily optimized, and (3) list indexing is just array indexing plus a little more work, so list indexing must be slightly slower.

As you have discovered, when you balance out one very fast thing against a great many slightly slower things, there comes a point where one will win over the other.

If this is the case, then it means than all code written in the world is wrong.

No it does not.

We should have predicted for this case.

No, we should not have worried about it. Don't believe me? Just find me a product in the marketplace today whose success or failure was predicated entirely upon choosing the list iteration technique that was a couple milliseconds faster.

And each "For" loop on a list should automatically call .ToArray before it starts.

Absolutely not. Even if we knew it was faster, which we don't, there are many kinds of performance to worry about. Doubling the amount of memory in use in order to save a couple milliseconds works directly against the performance goal of minimizing memory usage!  Not everyone cares about raw speed.

In your case the difference was 4000 milliseconds over 50000 runs, so that's less than a tenth of a millisecond saved on each run. If you care about that tenth of a millisecond, then by all means make this change to your code.

Body:
List Benchmark: Size=1000, Runs=50000, Total Time = 19.5 sec

var list = new List<string>(Size);
for (int i = 0; i < Size; i++) list.Add(i.ToString());

var b = new Benchmark();
b.Test("TestListIteration", () =>
{
   c = 0;
   for (int i = 0; i < Runs; i++)
   {
      for (int j = 0; j < Size; j++)
      {
          c += list[j].Length;
      }
   }
});

List To Array Benchmark: Size=1000, Runs=50000, Total Time=15.449

var list = new List<string>(Size);
for (int i = 0; i < Size; i++) list.Add(i.ToString());

var b = new Benchmark();
b.Test("TestListIteration", () =>
{
   c = 0;
   for (int i = 0; i < Runs; i++)
   {
      var array = list.ToArray(); //Changed line here !!!
      for (int j = 0; j < Size; j++)
      {
          c += array[j].Length;
      }
   }
});

Isn't this a paradox ?

How can performing two actions

  a) converting entire list to array and

  b) iterating entire array.

Be faster than doing b alone (iterating list).

If this is the case, then it means than all code written in the world is wrong. We should have predicted for this case. And each "For" loop on a list should automatically call .ToArray before it starts. Even if the array is discarded later.

Edit: Here are the Results depending on the "Size".

Size=10, Runs=5000000: List Wins
List : 20.362, ListToArray: 37.36

Size=100, Runs=500000: List Wins
List : 19.64, ListToArray: 23.162

Size=1000, Runs=50000: ListToArray Wins
List : 19.5, ListToArray: 15.449

Size=10000, Runs=5000: ListToArray Wins
List : 20.094, ListToArray: 14.453

Size=10000000, Runs=5: Computer died


问:
MKMapView GPS location question
I am a newbie.. I have a doubt in MapView and CoreLocation. How to make the mapview to display "blue dot and circle animation" as in Locate Me action of Maps app? The solution may be very simple, but I don't have much idea about map in iPhone. Help me, pls.</p>

答:
Try this

```<code>yourMapView.showsUserLocation=YES;
</code>```

Put this in your viewforAnnotation Delegate

```<code>if (annotation == mapView.userLocation)
{
    //You can customize the blue dot current location annotation here
    return nil;
}
</code>```

Title: MKMapView GPS location question

Body:
Try this

yourMapView.showsUserLocation=YES;

Put this in your viewforAnnotation Delegate

if (annotation == mapView.userLocation)
{
    //You can customize the blue dot current location annotation here
    return nil;
}