问:
Extract the best parameters from cva.glmnet object
I'm sure there is an elegant way to extract the best alpha and lambda after running cva.glmnet but somehow I cannot find it.

Here is the code I am using in the meantime.

Thank you

```<code>library(data.table);library(glmnetUtils);library(useful)

# make some dummy data

data(iris)

x <- useful::build.x(data = iris,formula = Sepal.Length ~ .)
y <- iris$Sepal.Length

# run cv for alpha in c(0,0.5,1)

output.of.cva.glmnet <- cva.glmnet(x=x,y=y,alpha = c(0,0.5,1))

# extract the best parameters

number.of.alphas.tested <- length(output.of.cva.glmnet$alpha)

cv.glmnet.dt <- data.table()

for (i in 1:number.of.alphas.tested){
  glmnet.model <- output.of.cva.glmnet$modlist[[i]]
  min.mse <-  min(glmnet.model$cvm)
  min.lambda <- glmnet.model$lambda.min
  alpha.value <- output.of.cva.glmnet$alpha[i]
  new.cv.glmnet.dt <- data.table(alpha=alpha.value,min_mse=min.mse,min_lambda=min.lambda)
  cv.glmnet.dt <- rbind(cv.glmnet.dt,new.cv.glmnet.dt)
}

best.params <- cv.glmnet.dt[which.min(cv.glmnet.dt$min_mse)]
</code>```

答:
Based on a thread I read on GitHub the author wants people to use plot(fit) instead of just outputting the best parameters. However, that isn't always possible, especially when cross validation is involved. These helper functions can be a good workaround.

```<code># Train model.
fit <- cva.glmnet(X, y)

# Get alpha.
get_alpha <- function(fit) {
  alpha <- fit$alpha
  error <- sapply(fit$modlist, function(mod) {min(mod$cvm)})
  alpha[which.min(error)]
}

# Get all parameters.
get_model_params <- function(fit) {
  alpha <- fit$alpha
  lambdaMin <- sapply(fit$modlist, `[[`, "lambda.min")
  lambdaSE <- sapply(fit$modlist, `[[`, "lambda.1se")
  error <- sapply(fit$modlist, function(mod) {min(mod$cvm)})
  best <- which.min(error)
  data.frame(alpha = alpha[best], lambdaMin = lambdaMin[best],
             lambdaSE = lambdaSE[best], eror = error[best])
}

</code>```

Body:
I'm sure there is an elegant way to extract the best alpha and lambda after running cva.glmnet but somehow I cannot find it.

Here is the code I am using in the meantime.

Thank you

library(data.table);library(glmnetUtils);library(useful)

# make some dummy data

data(iris)

x <- useful::build.x(data = iris,formula = Sepal.Length ~ .)
y <- iris$Sepal.Length

# run cv for alpha in c(0,0.5,1)

output.of.cva.glmnet <- cva.glmnet(x=x,y=y,alpha = c(0,0.5,1))

# extract the best parameters

number.of.alphas.tested <- length(output.of.cva.glmnet$alpha)

cv.glmnet.dt <- data.table()

for (i in 1:number.of.alphas.tested){
  glmnet.model <- output.of.cva.glmnet$modlist[[i]]
  min.mse <-  min(glmnet.model$cvm)
  min.lambda <- glmnet.model$lambda.min
  alpha.value <- output.of.cva.glmnet$alpha[i]
  new.cv.glmnet.dt <- data.table(alpha=alpha.value,min_mse=min.mse,min_lambda=min.lambda)
  cv.glmnet.dt <- rbind(cv.glmnet.dt,new.cv.glmnet.dt)
}

best.params <- cv.glmnet.dt[which.min(cv.glmnet.dt$min_mse)]

Body:
Based on a thread I read on GitHub the author wants people to use plot(fit) instead of just outputting the best parameters. However, that isn't always possible, especially when cross validation is involved. These helper functions can be a good workaround.

# Train model.
fit <- cva.glmnet(X, y)

# Get alpha.
get_alpha <- function(fit) {
  alpha <- fit$alpha
  error <- sapply(fit$modlist, function(mod) {min(mod$cvm)})
  alpha[which.min(error)]
}

# Get all parameters.
get_model_params <- function(fit) {
  alpha <- fit$alpha
  lambdaMin <- sapply(fit$modlist, `[[`, "lambda.min")
  lambdaSE <- sapply(fit$modlist, `[[`, "lambda.1se")
  error <- sapply(fit$modlist, function(mod) {min(mod$cvm)})
  best <- which.min(error)
  data.frame(alpha = alpha[best], lambdaMin = lambdaMin[best],
             lambdaSE = lambdaSE[best], eror = error[best])
}


来源: stackexchange

问:
cannot install cordova-plugin-wkwebview-engine
Long story short. Executing cordova plugin add cordova-plugin-wkwebview-engine
and getting Failed to fetch plugin cordova-plugin-wkwebview-engine via registry.
Probably this is either a connection problem, or plugin spec is incorrect.

What am I doing wrong?

答:
Please try the following workaround:

Execute this command: sudo npm install git+https://github.com/apache/cordova-plugin-wkwebview-engine.git#master --production --save, enter your password if required.

Then try the plugin add command: cordova plugin add cordova-plugin-wkwebview-engine.

ContentLicense: CC BY-SA 4.0


问:
Compare 2 CSV Huge CSV Files and print the differences to another csv file using perl
I have 2 csv files of multiple fields(approx 30 fields), and huge size ( approx 4GB ).

```<code>File1:

EmployeeName,Age,Salary,Address
Vinoth,12,2548.245,"140,North Street,India"
Vivek,40,2548.245,"140,North Street,India"
Karthick,10,10.245,"140,North Street,India"

File2:

EmployeeName,Age,Salary,Address
Vinoth,12,2548.245,"140,North Street,USA"
Karthick,10,10.245,"140,North Street,India"
Vivek,40,2548.245,"140,North Street,India"
</code>```

I want to compare these 2 files and report the differences into another csv file. In the above example, Employee Vivek and Karthick details are present in different row numbers but still the record data is same, so it should be considered as match. Employee Vinoth record should be considered as a mismatch since there is a mismatch in the address.

Output diff.csv file can contain the mismatched record from the File1 and File 2 as below.

```<code>Diff.csv
EmployeeName,Age,Salary,Address
F1, Vinoth,12,2548.245,"140,North Street,India"
F2, Vinoth,12,2548.245,"140,North Street,USA"
</code>```

I've written the code so far as below. After this I'm confused which option to choose whether a Binary Search or any other efficient way to do this. Could you please help me?

```<code>My approach
1. Load the File2 in memory as hashes of hashes.
2.Read line by line from File1 and match it with the hash of hashes in memory.

use strict;
use warnings;
use Text::CSV_XS;
use Getopt::Long;
use Data::Dumper;
use Text::CSV::Hashify;
use List::BinarySearch qw( :all );

# Get Command Line Parameters

my %opts = ();
GetOptions( \%opts, "file1=s", "file2=s", )
  or die("Error in command line arguments\n");

if ( !defined $opts{'file1'} ) {
    die "CSV file --file1 not specified.\n";
}
if ( !defined $opts{'file2'} ) {
    die "CSV file --file2 not specified.\n";
}

my $file1 = $opts{'file1'};
my $file2 = $opts{'file2'};
my $file3 = 'diff.csv';

print $file2 . "\n";

my $csv1 =
  Text::CSV_XS->new(
    { binary => 1, auto_diag => 1, sep_char => ',', eol => $/ } );
my $csv2 =
  Text::CSV_XS->new(
    { binary => 1, auto_diag => 1, sep_char => ',', eol => $/ } );
my $csvout =
  Text::CSV_XS->new(
    { binary => 1, auto_diag => 1, sep_char => ',', eol => $/ } );

open( my $fh1, '<:encoding(utf8)', $file1 )
  or die "Cannot not open '$file1' $!.\n";
open( my $fh2, '<:encoding(utf8)', $file2 )
  or die "Cannot not open '$file2' $!.\n";
open( my $fh3, '>:encoding(utf8)', $file3 )
  or die "Cannot not open '$file3' $!.\n";
binmode( STDOUT, ":utf8" );

my $f1line   = undef;
my $f2line   = undef;
my $header1  = undef;
my $f1empty  = 'false';
my $f2empty  = 'false';
my $reccount = 0;
my $hash_ref = hashify( "$file2", 'EmployeeName' );
if ( $f1empty eq 'false' ) {
    $f1line = $csv1->getline($fh1);
}
while (1) {

    if ( $f1empty eq 'false' ) {
        $f1line = $csv1->getline($fh1);
    }
    if ( !defined $f1line ) {
        $f1empty = 'true';
    }

    if ( $f1empty eq 'true' ) {
        last;
    }
    else {
        ## Read each line from File1 and match it with the File 2 which is loaded as hashes of hashes in perl. Need help here.

        }
    }

print "End of Program" . "\n";
</code>```

答:
```<code>#!/usr/bin/env perl

use Data::Dumper;
use Digest::MD5;
use 5.01800;
use warnings;

my %POS;
my %chars;

open my $FILEA,'<',q{FileA.txt}
    or die "Can't open 'FileA.txt' for reading! $!";
open my $FILEB,'<',q{FileB.txt}
    or die "Can't open 'FileB.txt' for reading! $!";
open my $OnlyInA,'>',q{OnlyInA.txt}
    or die "Can't open 'OnlyInA.txt' for writing! $!";
open my $InBoth,'>',q{InBoth.txt}
    or die "Can't open 'InBoth.txt' for writing! $!";
open my $OnlyInB,'>',q{OnlyInB.txt}
    or die "Can't open 'OnlyInB.txt' for writing! $!";
<$FILEA>,
    $POS{FILEA}=tell $FILEA;
<$FILEB>,
    $POS{FILEB}=tell $FILEB;
warn Data::Dumper->Dump([\%POS],[qw(*POS)]),' ';

{ # Scan for first character of the records involved
    while (<$FILEA>) {
        $chars{substr($_,0,1)}++;
        };
    while (<$FILEB>) {
        $chars{substr($_,0,1)}--;
        };
    # So what characters do we need to deal with?
    warn Data::Dumper->Dump([\%chars],[qw(*chars)]),' ';
    };
my @chars=sort keys %chars;
{
    my %_h;
    # For each of the characters in our character set
    for my $char (@chars) {
        warn Data::Dumper->Dump([\$char],[qw(*char)]),' ';
        # Beginning of data sections
        seek $FILEA,$POS{FILEA},0;
        seek $FILEB,$POS{FILEB},0;
        %_h=();
        my $pos=tell $FILEA;
        while (<$FILEA>) {
            next
                unless (substr($_,0,1) eq $char);
            # for each record save the lengthAndMD5 as the key and its start as the value
            $_h{lengthAndMD5(\$_)}=$pos;
            $pos=tell $FILEA;
            };
        my $_s;
        while (<$FILEB>) {
            next
                unless (substr($_,0,1) eq $char);
            if (exists $_h{$_s=lengthAndMD5(\$_)}) { # It's a duplicate
                print {$InBoth} $_;
                delete $_h{$_s};
                }
            else { # (Not in FILEA) It's only in FILEB
                print {$OnlyInB} $_;
                }
            };
        # only in FILEA
        warn Data::Dumper->Dump([\%_h],[qw(*_h)]),' ';
        for my $key (keys %_h) { # Only in FILEA
            seek $FILEA,delete $_h{$key},0;
            print {$OnlyInA} scalar <$FILEA>;
            };
        # Should be empty
        warn Data::Dumper->Dump([\%_h],[qw(*_h)]),' ';
        };
    };

close $OnlyInB
    or die "Could NOT close 'OnlyInB.txt' after writing! $!";
close $InBoth
    or die "Could NOT close 'InBoth.txt' after writing! $!";
close $OnlyInA
    or die "Could NOT close 'OnlyInA.txt' after writing! $!";
close $FILEB
    or die "Could NOT close 'FileB.txt' after reading! $!";
close $FILEA
    or die "Could NOT close 'FileA.txt' after reading! $!";
exit;

    sub lengthAndMD5 {
        return sprintf("%8.8lx-%32.32s",length(${$_[0]}),Digest::MD5::md5_hex(${$_[0]}));
        };

__END__
</code>```

Body:
I have 2 csv files of multiple fields(approx 30 fields), and huge size ( approx 4GB ).

File1:

EmployeeName,Age,Salary,Address
Vinoth,12,2548.245,"140,North Street,India"
Vivek,40,2548.245,"140,North Street,India"
Karthick,10,10.245,"140,North Street,India"

File2:

EmployeeName,Age,Salary,Address
Vinoth,12,2548.245,"140,North Street,USA"
Karthick,10,10.245,"140,North Street,India"
Vivek,40,2548.245,"140,North Street,India"

I want to compare these 2 files and report the differences into another csv file. In the above example, Employee Vivek and Karthick details are present in different row numbers but still the record data is same, so it should be considered as match. Employee Vinoth record should be considered as a mismatch since there is a mismatch in the address.

Output diff.csv file can contain the mismatched record from the File1 and File 2 as below.

Diff.csv
EmployeeName,Age,Salary,Address
F1, Vinoth,12,2548.245,"140,North Street,India"
F2, Vinoth,12,2548.245,"140,North Street,USA"

I've written the code so far as below. After this I'm confused which option to choose whether a Binary Search or any other efficient way to do this. Could you please help me?

My approach
1. Load the File2 in memory as hashes of hashes.
2.Read line by line from File1 and match it with the hash of hashes in memory.

use strict;
use warnings;
use Text::CSV_XS;
use Getopt::Long;
use Data::Dumper;
use Text::CSV::Hashify;
use List::BinarySearch qw( :all );

# Get Command Line Parameters

my %opts = ();
GetOptions( \%opts, "file1=s", "file2=s", )
  or die("Error in command line arguments\n");

if ( !defined $opts{'file1'} ) {
    die "CSV file --file1 not specified.\n";
}
if ( !defined $opts{'file2'} ) {
    die "CSV file --file2 not specified.\n";
}

my $file1 = $opts{'file1'};
my $file2 = $opts{'file2'};
my $file3 = 'diff.csv';

print $file2 . "\n";

my $csv1 =
  Text::CSV_XS->new(
    { binary => 1, auto_diag => 1, sep_char => ',', eol => $/ } );
my $csv2 =
  Text::CSV_XS->new(
    { binary => 1, auto_diag => 1, sep_char => ',', eol => $/ } );
my $csvout =
  Text::CSV_XS->new(
    { binary => 1, auto_diag => 1, sep_char => ',', eol => $/ } );

open( my $fh1, '<:encoding(utf8)', $file1 )
  or die "Cannot not open '$file1' $!.\n";
open( my $fh2, '<:encoding(utf8)', $file2 )
  or die "Cannot not open '$file2' $!.\n";
open( my $fh3, '>:encoding(utf8)', $file3 )
  or die "Cannot not open '$file3' $!.\n";
binmode( STDOUT, ":utf8" );

my $f1line   = undef;
my $f2line   = undef;
my $header1  = undef;
my $f1empty  = 'false';
my $f2empty  = 'false';
my $reccount = 0;
my $hash_ref = hashify( "$file2", 'EmployeeName' );
if ( $f1empty eq 'false' ) {
    $f1line = $csv1->getline($fh1);
}
while (1) {

    if ( $f1empty eq 'false' ) {
        $f1line = $csv1->getline($fh1);
    }
    if ( !defined $f1line ) {
        $f1empty = 'true';
    }

    if ( $f1empty eq 'true' ) {
        last;
    }
    else {
        ## Read each line from File1 and match it with the File 2 which is loaded as hashes of hashes in perl. Need help here.

        }
    }

print "End of Program" . "\n";


问:
java elasticsearch HighLevelRestClient throw "java.net.SocketException: Connection reset" intermittently

Basic env:

Docker 19.0.3

Elastichsearch 7.6.2

elasticsearch java client 7.6.2 (belongs to spring boot starter data elasticsearch 2.3.0)

I was using docker elasticsearch for a spring boot application(also a docker container), and sometimes when i try to access to a page which is showing a chart by the result of aggregation, it just throws exception like this:

```<code>java.io.IOException: Connection reset
    at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:828)
    at org.elasticsearch.client.RestClient.performRequest(RestClient.java:248)
    at org.elasticsearch.client.RestClient.performRequest(RestClient.java:235)
    at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1514)
    at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1484)
    at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1454)
    at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:970)
    ......
    at jdk.internal.reflect.GeneratedMethodAccessor506.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:665)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
    at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
    at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.csrf.CsrfFilter.doFilterInternal(CsrfFilter.java:141)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)
    at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.session.web.http.SessionRepositoryFilter.doFilterInternal(SessionRepositoryFilter.java:141)
    at org.springframework.session.web.http.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:82)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1590)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.base/java.lang.Thread.run(Thread.java:830)
Caused by: java.net.SocketException: Connection reset
    at java.base/sun.nio.ch.SocketChannelImpl.throwConnectionReset(SocketChannelImpl.java:345)
    at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:376)
    at org.apache.http.impl.nio.reactor.SessionInputBufferImpl.fill(SessionInputBufferImpl.java:231)
    at org.apache.http.impl.nio.codecs.AbstractMessageParser.fillBuffer(AbstractMessageParser.java:136)
    at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:241)
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
    at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
    ... 1 common frames omitted
</code>```

This is my spring es config:

```<code>spring:
  elasticsearch:
    rest:
      uris: http://elasticsearch:9200
      connection-timeout: 10s
</code>```

This is the java code I am using for agg:

```<code>        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
            .size(0)
            .timeout(TimeValue.timeValueSeconds(30))
            ... // some code just for setting query

        SearchRequest searchRequest = Requests
            .searchRequest(IncidentConstants.INCIDENT_INDICES)
            .source(searchSourceBuilder);
        SearchResponse res = esClient.search(searchRequest, RequestOptions.DEFAULT);
        ... // parse response
</code>```

It is a very simple query, but it just happens intermittently, at first i thought it was because of query timeout, but after I add the line .timeout(TimeValue.timeValueSeconds(30)), still not working

Hope someone can help me with this strange problem

答:
I found the reason, i was deploying entire env in swarm mode with an overlay network,

  and according to this post: https://success.docker.com/article/ipvs-connection-timeout-issue,

  it seems like overlay network tcp connection will be closed if there's no interaction in 15 minutes,

  and that's why it always throws connection reset every 15 minutes(since last time access to that query page), I found a solution after I take a deep look at code of spring data es,  codes:

```<code>@Configuration
public class QueryConfig {
    @Bean
    RestClientBuilderCustomizer keepAliveCustomizer() {
        return new KeepAliveCustomizer();
    }

    static class KeepAliveCustomizer implements RestClientBuilderCustomizer {
        @Override
        public void customize(RestClientBuilder builder) {

        }

        @Override
        public void customize(HttpAsyncClientBuilder builder) {
            builder.setKeepAliveStrategy((response, context) -> Duration.ofMinutes(5).toMillis());
        }

        @Override
        public void customize(RequestConfig.Builder builder) {
        }
    }
    ......
}
</code>```

So since the connection will be closed in 15 minutes, we just need to set the HttpClient's KeepAliveStrategy to let every connection in pool timeout in less that 15 minutes, what I set is 5 minutes

  The default KeepAliveStrategy is trying to get timeout from http response, and it is not working as I expected(didn't check if es rest api will return timeout header)

Body:
Basic env:

Docker 19.0.3

Elastichsearch 7.6.2

elasticsearch java client 7.6.2 (belongs to spring boot starter data elasticsearch 2.3.0)

I was using docker elasticsearch for a spring boot application(also a docker container), and sometimes when i try to access to a page which is showing a chart by the result of aggregation, it just throws exception like this:

java.io.IOException: Connection reset
    at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:828)
    at org.elasticsearch.client.RestClient.performRequest(RestClient.java:248)
    at org.elasticsearch.client.RestClient.performRequest(RestClient.java:235)
    at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1514)
    at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1484)
    at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1454)
    at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:970)
    ......
    at jdk.internal.reflect.GeneratedMethodAccessor506.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:665)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
    at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
    at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.csrf.CsrfFilter.doFilterInternal(CsrfFilter.java:141)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)
    at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.session.web.http.SessionRepositoryFilter.doFilterInternal(SessionRepositoryFilter.java:141)
    at org.springframework.session.web.http.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:82)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1590)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.base/java.lang.Thread.run(Thread.java:830)
Caused by: java.net.SocketException: Connection reset
    at java.base/sun.nio.ch.SocketChannelImpl.throwConnectionReset(SocketChannelImpl.java:345)
    at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:376)
    at org.apache.http.impl.nio.reactor.SessionInputBufferImpl.fill(SessionInputBufferImpl.java:231)
    at org.apache.http.impl.nio.codecs.AbstractMessageParser.fillBuffer(AbstractMessageParser.java:136)
    at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:241)
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
    at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
    ... 1 common frames omitted

This is my spring es config:

spring:
  elasticsearch:
    rest:
      uris: http://elasticsearch:9200
      connection-timeout: 10s

This is the java code I am using for agg:

        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
            .size(0)
            .timeout(TimeValue.timeValueSeconds(30))
            ... // some code just for setting query

        SearchRequest searchRequest = Requests
            .searchRequest(IncidentConstants.INCIDENT_INDICES)
            .source(searchSourceBuilder);
        SearchResponse res = esClient.search(searchRequest, RequestOptions.DEFAULT);
        ... // parse response

It is a very simple query, but it just happens intermittently, at first i thought it was because of query timeout, but after I add the line .timeout(TimeValue.timeValueSeconds(30)), still not working

Hope someone can help me with this strange problem

Body:
I found the reason, i was deploying entire env in swarm mode with an overlay network,

  and according to this post: https://success.docker.com/article/ipvs-connection-timeout-issue,

  it seems like overlay network tcp connection will be closed if there's no interaction in 15 minutes,

  and that's why it always throws connection reset every 15 minutes(since last time access to that query page), I found a solution after I take a deep look at code of spring data es,  codes:

@Configuration
public class QueryConfig {
    @Bean
    RestClientBuilderCustomizer keepAliveCustomizer() {
        return new KeepAliveCustomizer();
    }

    static class KeepAliveCustomizer implements RestClientBuilderCustomizer {
        @Override
        public void customize(RestClientBuilder builder) {

        }

        @Override
        public void customize(HttpAsyncClientBuilder builder) {
            builder.setKeepAliveStrategy((response, context) -> Duration.ofMinutes(5).toMillis());
        }

        @Override
        public void customize(RequestConfig.Builder builder) {
        }
    }
    ......
}

So since the connection will be closed in 15 minutes, we just need to set the HttpClient's KeepAliveStrategy to let every connection in pool timeout in less that 15 minutes, what I set is 5 minutes

  The default KeepAliveStrategy is trying to get timeout from http response, and it is not working as I expected(didn't check if es rest api will return timeout header)


问:
How to get numbers of variants numerical combinations in javascript?
I need to implement a function that accepts two parameters - the number of 0 and the number of 1 and determine how many ways to place these 0 and 1 so that there are no two zeros in a row.

For example, I need to find all methods of placing two 0 and two 1.

There are six possible ways to place them: 0011, 0101, 0110, 1001, 1010, 1100.

In three cases there are two zeros in a row: 0011, 1001 and 1100.

I subtract them from the total number and get three possible ways: 0101, 0110 and 1010. So the answer is 3.

First, I'm trying to write script to recognize which cases I need

let arr = ["1100","1010","1001","0011","0101","0110"]
let result = [];
for (let i = 0; i < arr.length; i++){
  let expVal = arr[i];
  for (let p = 0; p < expVal.length; p++){
    if (expVal[p] === expVal[p++] || expVal[p] === "0"){
      result.push(expVal)
    }
  }
}
console.log(result);```

It's does not work. I don't know how to fix it.
And I don't understand what I need to do later

答:
The problem you are solving is equivalent to the Fibonacci sequence.

filter for includes '00'

recursive function x generates binary number strings using n0 zeroes, n1 ones.  Works by branching to (add a zero, find combos for one less zero)'0'+x(n0-1,n1) and (add a one, find combos for one less 1)'1'+x(n0,n1-1).

let arr = ["1100","1010","1001","0011","0101","0110"]

const x = (n0,n1) =>
  !(n0 === 0 || n1 === 0) ?
    x(n0-1,n1).map(x=>'0'+x).concat(
    x(n0,n1-1).map(x=>'1'+x))
    : ['0'.repeat(n0)||'1'.repeat(n1)]

arr = x(2,2)

console.log(
x(2,2)
)

console.log(arr.filter(x=>x.includes('00')))```

Body:
I need to implement a function that accepts two parameters - the number of 0 and the number of 1 and determine how many ways to place these 0 and 1 so that there are no two zeros in a row.

For example, I need to find all methods of placing two 0 and two 1.

There are six possible ways to place them: 0011, 0101, 0110, 1001, 1010, 1100.

In three cases there are two zeros in a row: 0011, 1001 and 1100.

I subtract them from the total number and get three possible ways: 0101, 0110 and 1010. So the answer is 3.

First, I'm trying to write script to recognize which cases I need

let arr = ["1100","1010","1001","0011","0101","0110"]
let result = [];
for (let i = 0; i < arr.length; i++){
  let expVal = arr[i];
  for (let p = 0; p < expVal.length; p++){
    if (expVal[p] === expVal[p++] || expVal[p] === "0"){
      result.push(expVal)
    }
  }
}
console.log(result);

It's does not work. I don't know how to fix it.
And I don't understand what I need to do later


问:
Python and using data from Google Sheets
I'm wondering what's the best way to analyse data being pulled from Google Sheets. I've successfully managed to connect to a specific sheet and return all data using:

```<code>scope = ['https://spreadsheets.google.com/feeds','https://www.googleapis.com/auth/drive']
creds = ServiceAccountCredentials.from_json_keyfile_name('client_secret.json', scope)
client = gspread.authorize(creds)

sheet = client.open('KB Swings').sheet1

total_swings = sheet.get_all_records()
print(total_swings)

</code>```

But I want to do a basic sum of data in a specific column.

答:
I believe this is what you want but have you considered computing the sum in your spreadsheet?

    #get column values in column n
    col_vals = sheet.col_values(n)
    # sum of list
    col_sum = sum(col_vals)
```

Body:
I'm wondering what's the best way to analyse data being pulled from Google Sheets. I've successfully managed to connect to a specific sheet and return all data using:

scope = ['https://spreadsheets.google.com/feeds','https://www.googleapis.com/auth/drive']
creds = ServiceAccountCredentials.from_json_keyfile_name('client_secret.json', scope)
client = gspread.authorize(creds)

sheet = client.open('KB Swings').sheet1

total_swings = sheet.get_all_records()
print(total_swings)

But I want to do a basic sum of data in a specific column.


问:
Creating train/test/val split with StratifiedKFold
I'm trying to use StratifiedKFold to create train/test/val splits for use in a non-sklearn machine learning work flow. So, the DataFrame needs to be split and then stay that way.

I'm trying to do it like the following, using .values because I'm passing pandas DataFrames:

```<code>skf = StratifiedKFold(n_splits=3, shuffle=False)
skf.get_n_splits(X, y)

for train_index, test_index, valid_index in skf.split(X.values, y.values):
    print("TRAIN:", train_index, "TEST:", test_index,  "VALID:", valid_index)
    X_train, X_test, X_valid = X.values[train_index], X.values[test_index], X.values[valid_index]
    y_train, y_test, y_valid = y.values[train_index], y.values[test_index], y.values[valid_index]
</code>```

This fails with:

ValueError: not enough values to unpack (expected 3, got 2).
```

<p>I read through all of the <code>sklearn</code> docs and ran the example code, but did not gain a better understanding of how to use stratified k fold splits outside of a <code>sklearn</code> cross-validation scenario. </p>

<p>EDIT:</p>

<p>I also tried like this:</p>

```# Create train/test split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1, stratify=y)

# Create validation split from train split
X_train, X_valid, y_train, y_valid = train_test_split(X_train, y_train, test_size=0.05)
```

Which seems to work, although I imagine I'm messing with the stratification by doing so.

答:
I'm not exactly sure if this question is about KFold or just stratified splits, but I wrote this quick wrapper for StratifiedKFold with a cross validation set.

from sklearn.model_selection import StratifiedKFold, train_test_split

class StratifiedKFold3(StratifiedKFold):

    def split(self, X, y, groups=None):
        s = super().split(X, y, groups)
        for train_indxs, test_indxs in s:
            y_train = y[train_indxs]
            train_indxs, cv_indxs = train_test_split(train_indxs,stratify=y_train, test_size=(1 / (self.n_splits - 1)))
            yield train_indxs, cv_indxs, test_indxs
```

<p>It can be used like this:</p>

<pre class="lang-py prettyprint-override"><code>X = np.random.rand(100)
y = np.random.choice([0,1],100)
g = KFold3(10).split(X,y)
train, cv, test = next(g)
train.shape, cv.shape, test.shape
>> ((80,), (10,), (10,))
</code>```

ContentLicense: CC BY-SA 3.0

Body:
I'm trying to use StratifiedKFold to create train/test/val splits for use in a non-sklearn machine learning work flow. So, the DataFrame needs to be split and then stay that way.

I'm trying to do it like the following, using .values because I'm passing pandas DataFrames:

skf = StratifiedKFold(n_splits=3, shuffle=False)
skf.get_n_splits(X, y)

for train_index, test_index, valid_index in skf.split(X.values, y.values):
    print("TRAIN:", train_index, "TEST:", test_index,  "VALID:", valid_index)
    X_train, X_test, X_valid = X.values[train_index], X.values[test_index], X.values[valid_index]
    y_train, y_test, y_valid = y.values[train_index], y.values[test_index], y.values[valid_index]

This fails with:

ValueError: not enough values to unpack (expected 3, got 2).

I read through all of the sklearn docs and ran the example code, but did not gain a better understanding of how to use stratified k fold splits outside of a sklearn cross-validation scenario.

EDIT:

I also tried like this:

# Create train/test split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1, stratify=y)

# Create validation split from train split
X_train, X_valid, y_train, y_valid = train_test_split(X_train, y_train, test_size=0.05)

Which seems to work, although I imagine I'm messing with the stratification by doing so.

Body:
I'm not exactly sure if this question is about KFold or just stratified splits, but I wrote this quick wrapper for StratifiedKFold with a cross validation set.

from sklearn.model_selection import StratifiedKFold, train_test_split

class StratifiedKFold3(StratifiedKFold):

    def split(self, X, y, groups=None):
        s = super().split(X, y, groups)
        for train_indxs, test_indxs in s:
            y_train = y[train_indxs]
            train_indxs, cv_indxs = train_test_split(train_indxs,stratify=y_train, test_size=(1 / (self.n_splits - 1)))
            yield train_indxs, cv_indxs, test_indxs

It can be used like this:

X = np.random.rand(100)
y = np.random.choice([0,1],100)
g = KFold3(10).split(X,y)
train, cv, test = next(g)
train.shape, cv.shape, test.shape
>> ((80,), (10,), (10,))


问:
OpenXML - left table border doesn't work?
Here is my border formatting for creating a test table in a docx using OpenXML (C#):

```<code>    //Create a TableProperties object and specify its border information.
    var borderType = BorderValues.Single;
    DocumentFormat.OpenXml.UInt32Value size = 5;
    TableProperties tblProp = new TableProperties(
        new TableJustification() { Val = TableRowAlignmentValues.Center },
        new TableBorders(
            new TopBorder(){ Val = new EnumValue<BorderValues>(borderType), Size = size },
            new BottomBorder(){ Val = new EnumValue<BorderValues>(borderType), Size = size },
            new LeftBorder(){ Val = new EnumValue<BorderValues>(borderType), Size = size },
            new RightBorder(){ Val = new EnumValue<BorderValues>(borderType), Size = size },
            new InsideHorizontalBorder(){ Val = new EnumValue<BorderValues>(borderType), Size = size },
            new InsideVerticalBorder(){ Val = new EnumValue<BorderValues>(borderType), Size = size }
        )
    );

    // Append the TableProperties object to the empty table.
    table.AppendChild<TableProperties>(tblProp);
</code>```

The above formatting produces this (when viewed in 365 word viewer):

  The question is, why?

Here is the generated xml (extracted from zip):

EDIT: it renders correctly in the word desktop app.. so this seems to be a #Microsoft bug.

答:
Different clients often support the specification to different degrees. I remember fighting this when I was trying to produce content for Google docs.

There are some things you can try.

Firstly, instead of using LeftBorder and RightBorder, try StartBorder and EndBorder. These are important to support semantic considerations in RTL cultures vs. LTR cultures, and may consequently be better supported.

Secondly, reverse engineering nearly always worked for me. Start by styling a document how you want it to look in Word or other tools, then extract and examine the XML.

I think you'll find that Word defines a style that it applies to the table instead of trying to do it all inline - copying that approach is more complex, but often does the trick.

Body:
Here is my border formatting for creating a test table in a docx using OpenXML (C#):

    //Create a TableProperties object and specify its border information.
    var borderType = BorderValues.Single;
    DocumentFormat.OpenXml.UInt32Value size = 5;
    TableProperties tblProp = new TableProperties(
        new TableJustification() { Val = TableRowAlignmentValues.Center },
        new TableBorders(
            new TopBorder(){ Val = new EnumValue<BorderValues>(borderType), Size = size },
            new BottomBorder(){ Val = new EnumValue<BorderValues>(borderType), Size = size },
            new LeftBorder(){ Val = new EnumValue<BorderValues>(borderType), Size = size },
            new RightBorder(){ Val = new EnumValue<BorderValues>(borderType), Size = size },
            new InsideHorizontalBorder(){ Val = new EnumValue<BorderValues>(borderType), Size = size },
            new InsideVerticalBorder(){ Val = new EnumValue<BorderValues>(borderType), Size = size }
        )
    );

    // Append the TableProperties object to the empty table.
    table.AppendChild<TableProperties>(tblProp);

The above formatting produces this (when viewed in 365 word viewer):

  The question is, why?

Here is the generated xml (extracted from zip):

EDIT: it renders correctly in the word desktop app.. so this seems to be a #Microsoft bug.


问:
Themeable browser use UIWebView forcefully with ionic capacitor
I am doing ionic application with capacitor.

I have to use https://ionicframework.com/docs/native/themeable-browser plugin for one funcationality but when I make build and upload to apple store app is rejected because of UiWebView issue.

Commands for installing themeable plugin

```<code>npm install cordova-plugin-themeablebrowser
npm install @ionic-native/themeable-browser
ionic cap sync
</code>```

If I remove that plugin and upload it to apple build process pass successfully.

答:
Apple no longer accepts new submissions using UIWebView:
https://forums.developer.apple.com/thread/122114

There is also an Ionic blog article: https://ionicframework.com/blog/understanding-itms-90809-uiwebview-api-deprecation/

You can use Capacitor's Browser API, but it is quite limited compared to some existing Cordova plugins. However it is built around WKWebView and is therefore compliant.

There is AFAIK no alternative but to move to WKWebView if you want your app to be on the Apple App Store.

Body:
I am doing ionic application with capacitor.

I have to use https://ionicframework.com/docs/native/themeable-browser plugin for one funcationality but when I make build and upload to apple store app is rejected because of UiWebView issue.

Commands for installing themeable plugin

npm install cordova-plugin-themeablebrowser
npm install @ionic-native/themeable-browser
ionic cap sync

If I remove that plugin and upload it to apple build process pass successfully.


问:
UML help C# Design Principles
Could anyone please point out the meaning of the graph below:

What is the relationship between PolicyLayer and PolicyServiceInterface

What is the relationship between PolicyServiceInterface and MachanismLayer.

C# code would be also appreciated!

Please note that the UML is from Agile Principles, Patterns, and Practices in C#
By Martin C. Robert, Martin Micah 2006.

Added on 15 2011/6/2

Do the following have the same meaning:
1) A solid line with a triangle at one end
2) A dashed line with a triangle at one end

Added on 2011/6/3  1st

What is the difference between:
1) A solid line with an arrow at one end
2) A dashed line with an arrow at one end

Example in thie sample and PersistentObject and ThirdPartyPersistentSet in the link below:

UML help C# Design Principles

Added on 2011/6/3 2nd

Can the relationship between PolicyLayer and PolicyServiceInterface as below:

```<code>public class PolicyLayer

{
    private PolicyServiceInterface policyServiceInterface = new PolicyServiceInterfaceImplementation();
}

class PolicyServiceInterfaceImplementation:PolicyServiceInterface {}
</code>```

Regarding

答:
PolicyLayer uses Policy service interface(probabily it holds a reference)
MachanismLayer implements PolicyServiceInterface </p>

```<code> public interface IPolicyServiceInterface
    {
        void DoSomething();
    }

    public class MachanismLayer : IPolicyServiceInterface
    {
        public void DoSomething()
        {
            Console.WriteLine("MachanismLayer Do Something");
        }
    }

    public class PolicyLayer
    {
        private IPolicyServiceInterface _policyServiceInterface;
        public PolicyLayer(IPolicyServiceInterface policyServiceInterface)
        {
            _policyServiceInterface = policyServiceInterface;
        }

        public void DoSomethig()
        {
            _policyServiceInterface.DoSomething();
        }
    }

    public class Program
    {
        public static void Main(string[] agrs)
        {
           MachanismLayer machanismLayer=new MachanismLayer();
           PolicyLayer policyLayer = new PolicyLayer(machanismLayer);
           policyLayer.DoSomethig();
        }
    }
</code>```

Title: UML help C# Design Principles

Body:
Could anyone please point out the meaning of the graph below:

What is the relationship between PolicyLayer and PolicyServiceInterface

What is the relationship between PolicyServiceInterface and MachanismLayer.

C# code would be also appreciated!

Please note that the UML is from Agile Principles, Patterns, and Practices in C#
By Martin C. Robert, Martin Micah 2006.

Added on 15 2011/6/2

Do the following have the same meaning:
1) A solid line with a triangle at one end
2) A dashed line with a triangle at one end

Added on 2011/6/3  1st

What is the difference between:
1) A solid line with an arrow at one end
2) A dashed line with an arrow at one end

Example in thie sample and PersistentObject and ThirdPartyPersistentSet in the link below:

UML help C# Design Principles

Added on 2011/6/3 2nd

Can the relationship between PolicyLayer and PolicyServiceInterface as below:

public class PolicyLayer

{
    private PolicyServiceInterface policyServiceInterface = new PolicyServiceInterfaceImplementation();
}

class PolicyServiceInterfaceImplementation:PolicyServiceInterface {}

Regarding

Body:
PolicyLayer uses Policy service interface(probabily it holds a reference)
MachanismLayer implements PolicyServiceInterface

 public interface IPolicyServiceInterface
    {
        void DoSomething();
    }

    public class MachanismLayer : IPolicyServiceInterface
    {
        public void DoSomething()
        {
            Console.WriteLine("MachanismLayer Do Something");
        }
    }

    public class PolicyLayer
    {
        private IPolicyServiceInterface _policyServiceInterface;
        public PolicyLayer(IPolicyServiceInterface policyServiceInterface)
        {
            _policyServiceInterface = policyServiceInterface;
        }

        public void DoSomethig()
        {
            _policyServiceInterface.DoSomething();
        }
    }

    public class Program
    {
        public static void Main(string[] agrs)
        {
           MachanismLayer machanismLayer=new MachanismLayer();
           PolicyLayer policyLayer = new PolicyLayer(machanismLayer);
           policyLayer.DoSomethig();
        }
    }


问:
What is the best way of updating a variable in a view controller from scene delegate?
I am using Spotify SDK. I want to change labels in some view controllers when a user changes his/her player state. Here is my scene delegate:

```<code>var playerViewController = MatchViewController()

func playerStateDidChange(_ playerState: SPTAppRemotePlayerState) {
    playerViewController.stateChanged(playerState)
}
</code>```

A view controller:

```<code>func stateChanged(_ playerState: SPTAppRemotePlayerState) {
    // aLabel.text = playerState.track.name
}
</code>```

The problem is labels or other outlets are nil when the state is changed because the view controllers are not loaded at that time. How can I fix that? (I tried isViewLoaded)

答:
If you have a more than a few places to update according to a change that occurs at one place use observers. Here's how,

Post notification in SceneDelegate like this:

func playerStateDidChange(_ playerState: SPTAppRemotePlayerState) {
    NotificationCenter.default.post(name: NSNotification.Name(rawValue: "stateChanged"), object: nil, userInfo: ["playerState": playerState])
}
```

<p>Observe in ViewControllers like this:</p>

<pre class="lang-swift prettyprint-override"><code>class ViewController: UIViewController {
    override func viewDidLoad() {
        super.viewDidLoad()
        NotificationCenter.default.addObserver(self, selector: #selector(stateChanged), name: NSNotification.Name("stateChanged"), object: nil)
    }

    @objc func stateChanged(_ notification: Notification) {
        if let playerState = notification.userInfo?["playerState"] as? SPTAppRemotePlayerState {
            print(playerState)
        }
    }
}
</code>```

Body:
I am using Spotify SDK. I want to change labels in some view controllers when a user changes his/her player state. Here is my scene delegate:

var playerViewController = MatchViewController()

func playerStateDidChange(_ playerState: SPTAppRemotePlayerState) {
    playerViewController.stateChanged(playerState)
}

A view controller:

func stateChanged(_ playerState: SPTAppRemotePlayerState) {
    // aLabel.text = playerState.track.name
}

The problem is labels or other outlets are nil when the state is changed because the view controllers are not loaded at that time. How can I fix that? (I tried isViewLoaded)

Body:
If you have a more than a few places to update according to a change that occurs at one place use observers. Here's how,

Post notification in SceneDelegate like this:

func playerStateDidChange(_ playerState: SPTAppRemotePlayerState) {
    NotificationCenter.default.post(name: NSNotification.Name(rawValue: "stateChanged"), object: nil, userInfo: ["playerState": playerState])
}

Observe in ViewControllers like this:

class ViewController: UIViewController {
    override func viewDidLoad() {
        super.viewDidLoad()
        NotificationCenter.default.addObserver(self, selector: #selector(stateChanged), name: NSNotification.Name("stateChanged"), object: nil)
    }

    @objc func stateChanged(_ notification: Notification) {
        if let playerState = notification.userInfo?["playerState"] as? SPTAppRemotePlayerState {
            print(playerState)
        }
    }
}


问:
What are data breakpoints?
I just came to know that there are data breakpoints. I have worked for the last 5 years in C++ using Visual Studio, and I have never used data breakpoints.

Can someone throw some light on what data breakpoints are, when to use them and how to use them with VS?

As per my understanding we can set a data breakpoint when we want to check for changes to a variable's value. In this case, we can set a data breakpoint with a condition on the variable value.

Any other examples?

答:
Good ol' Daniel LeCheminant has a solid answer on what a data breakpoint does, so i'll toss in some anecdotes that highlight useful uses:

Any scenario where you know what will change, but have little or no idea where the code changing it lives (since otherwise you could simply use a conditional breakpoint). Specifically,

"Impossible" scenarios - program is crashing, because variable X is NULL, when variable X should never be NULL because no code anywhere ever sets variable X to NULL. Put a normal breakpoint in the code that initializes X, and when it is hit, set up a data breakpoint to watch for the change to NULL. Somewhat more common is the case where memory is released too early, and there are still pointers to it hanging around: use data breakpoints to find out who's releasing the memory.

Tedious scenarios - a 3rd-party library is doing bad, nasty, horrible things to your data structures. You know it's happening, because someone is trashing your data and obviously your code is perfect. But you don't know where, or when. Sure, you could single-step through a megabyte of disassembled DLL... but why bother, when you can set a data breakpoint on your data, sit back, and wait for it to get trashed!

Heisenbugs - similar to the impossible scenario, but they go away when you watch too closely, such that normal breakpoints - even conditional breakpoints - are useless. Timing and user-input sensitive logic is particularly vulnerable to this sort of thing. Since data breakpoints don't require the debugger to actually break at all until the time is right, assuming you can come up with a memory location that will only change when that elusive bug actually occurs you can use data breakpoints to set a trap for the Heisenbug and catch it in flagrante delicto.

Spaghetti scenarios - common in old, rotten code bases where global data is accessed everywhere. Yeah, you could use plain ol' conditional breakpoints... but you'd need hundreds of them. Data breakpoints make it easy.

ContentLicense: CC BY-SA 2.5

Title: What are data breakpoints?

LastEditorDisplayName: MarkJ


问:
Missing Request parameters with UTF-8 when javascript escape() is used
I am using jquery ajax api to submit (POST/GET) a text as parameter to a processing page.

```<code><input type="text" id="txtboxdata" name="txtboxdata" >

var dataString = "tmpVar="+ escape( $("#txtboxdata").val() );
$.ajax({ type: "POST",
                  url: "processing.jsp",
                  data: dataString ,
                  dataType: "json",
                  success: function(){}
        });

//dataString holds the parameter and value to be passed. Value is retrieved from a text box.
</code>```

If the text box

contains UTF-8 characters

and

apply the javascript "escape" on the text box value

then the parameter tmpVar goes disappears from the request object in the processing page(processing.jsp).

I used the debugger, and checked the request object. The parameter called "tmpVar" never shows up.

This works correctly when the following conditions are met

tmpVar shows up in the request object of processing.jsp when

there are no UTF-8 characters.

I do not apply "escape" before making the ajax call.

My Question
I would like to understand why does applying escape on a text containing UTF-8 not show up in the request object?

PS: I used "form serialize()" to solve the problem, just trying to understand why the issue occured.

Firebug: looks fine, this always shows up correctly. No missing data before the actual POST/GET.

答: You should call encodeURIComponent instead.

Body:
I am using jquery ajax api to submit (POST/GET) a text as parameter to a processing page.

<input type="text" id="txtboxdata" name="txtboxdata" >

var dataString = "tmpVar="+ escape( $("#txtboxdata").val() );
$.ajax({ type: "POST",
                  url: "processing.jsp",
                  data: dataString ,
                  dataType: "json",
                  success: function(){}
        });

//dataString holds the parameter and value to be passed. Value is retrieved from a text box.

If the text box

contains UTF-8 characters

and

apply the javascript "escape" on the text box value

then the parameter tmpVar goes disappears from the request object in the processing page(processing.jsp).

I used the debugger, and checked the request object. The parameter called "tmpVar" never shows up.

This works correctly when the following conditions are met

tmpVar shows up in the request object of processing.jsp when

there are no UTF-8 characters.

I do not apply "escape" before making the ajax call.

My Question
I would like to understand why does applying escape on a text containing UTF-8 not show up in the request object?

PS: I used "form serialize()" to solve the problem, just trying to understand why the issue occured.

Firebug: looks fine, this always shows up correctly. No missing data before the actual POST/GET.


问:
How to obtain public ip address using windows command prompt?
Any way of doing this through the command prompt without using any third party software or relying on powershell?

A simple command would be great like on Linux/Mac which I use curl http://ipinfo.io/ip for.

Thanks!

答:
This works perfectly for me: </p>

```<code>curl http://ip.jsontest.com/ > ip.json
for /f "tokens=2 delims=:" %%a in ('type ip.json') do (set publicip=%%a)
set publicip=%publicip:{=%
set publicip=%publicip:}=%
set publicip=%publicip:"=%
echo %publicip:~1%
</code>```

Body:
This works perfectly for me:

curl http://ip.jsontest.com/ > ip.json
for /f "tokens=2 delims=:" %%a in ('type ip.json') do (set publicip=%%a)
set publicip=%publicip:{=%
set publicip=%publicip:}=%
set publicip=%publicip:"=%
echo %publicip:~1%


问:
How to prevent direct access to private members outside of getters/setters in the same class?

Challenge

Given a class with a non-trivial getter and some other internal method accessing the same private member field:

Valid Example

class TestType {
    private String value;

    public String getValue() {
        return this.value == null || this.value.isEmpty() ? "default" : this.value;
    }

    public void setValue(String newValue) {
        this.value = newValue;
    }

    public int getValueLength() {
        return this.getValue().length();
    }
}
```

<p>I want to forbid the by-passing of the getter in the <code>getValueLength()</code> method.
The following implementation of <code>getValueLength()</code> should produce some kind of error (unit test failure, checkstyle error, or whatever else can be automated in a generic way):</p>

<h3>Invalid Method</h3>

<pre class="lang-java prettyprint-override"><code>    public int getValueLength() {
        // ERROR: possible NullPointerException
        return this.value.length();
    }
</code>```

Background

In my real-life code there is some lazy-loading going on inside the getter that only loads the actual value when it is being accessed for the first time. Calling getValueLength() first may result in a NullPointerException directly or perform wrong actions based on the missing value.

The motivation is to ensure that future developers don't forget to only use the getter method and never access the member directly. Since this is an isolated issue (i.e. only where this kind of lazy-loaded is being specifically added) it is acceptable if some extra annotations are required, e.g. on the member itself that it should only be accessed in selected methods and those methods to be marked as well -- or specific checkstyle comments enabling/disabling specific rules.

答:
Just to put this answer to form:

```<code>class LoadableVar<T> {

    private T val;
    private Supplier<T> loader;

    public LoadableVar(Supplier<? extends T> loader) {
        this.loader = loader;
    }

    public T get() {
        if (this.val == null) {
            //see: volatile and double-locking if multithreading
            this.val = this.loader.get();
        }
        return this.val;
    }

    public void set(T overwrite) { //WARN: ignores the loader!
        this.val = overwrite;
    }
}
</code>```

Then, in applying it to your class:

```<code>class TestType {
    private final LoadableVar<String> value;

    public TestType() {
        //can also be passed into the class, or done however you desire
        this.value = new LoadableVar<>(() -> /* load string from i/o, etc */);
    }

    public String getValue() {
        return this.value.get();
    }

    public void setValue(String newValue) {
        this.value.set(newValue); //I don't think this should be settable, personally
    }

    public int getValueLength() {
        return this.getValue().length();
    }
}
</code>```

Now when you are writing code within TestType's scope:

```<code>String s;
s = this.value; //compile error!
s = this.getValue(); //OK
s = this.value.get(); //OK
</code>```

As you can see, in doing it this way you've also made TestType#getValue redundant, you could simply allow value to be a protected member (and in which case, I'd remove those setters and make it immutable).

Body:
Challenge

Given a class with a non-trivial getter and some other internal method accessing the same private member field:

Valid Example

class TestType {
    private String value;

    public String getValue() {
        return this.value == null || this.value.isEmpty() ? "default" : this.value;
    }

    public void setValue(String newValue) {
        this.value = newValue;
    }

    public int getValueLength() {
        return this.getValue().length();
    }
}

I want to forbid the by-passing of the getter in the getValueLength() method.
The following implementation of getValueLength() should produce some kind of error (unit test failure, checkstyle error, or whatever else can be automated in a generic way):

Invalid Method

    public int getValueLength() {
        // ERROR: possible NullPointerException
        return this.value.length();
    }

Background

In my real-life code there is some lazy-loading going on inside the getter that only loads the actual value when it is being accessed for the first time. Calling getValueLength() first may result in a NullPointerException directly or perform wrong actions based on the missing value.

The motivation is to ensure that future developers don't forget to only use the getter method and never access the member directly. Since this is an isolated issue (i.e. only where this kind of lazy-loaded is being specifically added) it is acceptable if some extra annotations are required, e.g. on the member itself that it should only be accessed in selected methods and those methods to be marked as well -- or specific checkstyle comments enabling/disabling specific rules.

Body:
Just to put this answer to form:

class LoadableVar<T> {

    private T val;
    private Supplier<T> loader;

    public LoadableVar(Supplier<? extends T> loader) {
        this.loader = loader;
    }

    public T get() {
        if (this.val == null) {
            //see: volatile and double-locking if multithreading
            this.val = this.loader.get();
        }
        return this.val;
    }

    public void set(T overwrite) { //WARN: ignores the loader!
        this.val = overwrite;
    }
}

Then, in applying it to your class:

class TestType {
    private final LoadableVar<String> value;

    public TestType() {
        //can also be passed into the class, or done however you desire
        this.value = new LoadableVar<>(() -> /* load string from i/o, etc */);
    }

    public String getValue() {
        return this.value.get();
    }

    public void setValue(String newValue) {
        this.value.set(newValue); //I don't think this should be settable, personally
    }

    public int getValueLength() {
        return this.getValue().length();
    }
}

Now when you are writing code within TestType's scope:

String s;
s = this.value; //compile error!
s = this.getValue(); //OK
s = this.value.get(); //OK

As you can see, in doing it this way you've also made TestType#getValue redundant, you could simply allow value to be a protected member (and in which case, I'd remove those setters and make it immutable).


问:
Java 8 stream filter local variable
```<code>Obj1 x = null;
String key = null;
while(it.hasNext()){
   x = it.next();
   key = x.getKey();
   listObject.stream().filter(e -> e.getKey().equals(key)).findFirst().get();
}
</code>```

This is complaining with the following message: "Local variable key must be final or effective final..." I understand the the lambda context is created at runtime and it needs variables that do not change.

How can I use the functional style to find the object in the list?
I want to avoid the old fashion way of iterating over the list with while.

答:
In this case you would just need to make key final. For instance, this should work:</p>

```<code>while(it.hasNext()){
   Obj1 x = it.next();
   listObject.stream()
   .filter(e -> e.getKey().equals(x.getKey()))
   .findFirst().get();
}
</code>```

Title: Java 8 stream filter local variable

Body:
Obj1 x = null;
String key = null;
while(it.hasNext()){
   x = it.next();
   key = x.getKey();
   listObject.stream().filter(e -> e.getKey().equals(key)).findFirst().get();
}

This is complaining with the following message: "Local variable key must be final or effective final..." I understand the the lambda context is created at runtime and it needs variables that do not change.

How can I use the functional style to find the object in the list?
I want to avoid the old fashion way of iterating over the list with while.

Body:
In this case you would just need to make key final. For instance, this should work:

while(it.hasNext()){
   Obj1 x = it.next();
   listObject.stream()
   .filter(e -> e.getKey().equals(x.getKey()))
   .findFirst().get();
}


问:
Finding empty directories in Python
All,

What is the best way to check to see if there is data in a directory before deleting it? I am browsing through a couple pages to find some pics using wget and of course every page does not have an image on it but the directory is still created.

```<code>dir = 'Files\\%s' % (directory)
os.mkdir(dir)
cmd = 'wget -r -l1 -nd -np -A.jpg,.png,.gif -P %s %s' %(dir,  i[1])
os.system(cmd)
if not os.path.isdir(dir):
    os.rmdir(dir)
</code>```

I would like to test to see if a file was dropped in the directory after it was created. If nothing is there...delete it.

Thanks,
Adam

答:
I will go with EAFP like so:

```<code>try:
    os.rmdir(dir)
except OSError as ex:
    if ex.errno == errno.ENOTEMPTY:
        print "directory not empty"
</code>```

os.rmdir will not delete a directory that is not empty.

Title: Finding empty directories in Python

Body:
All,

What is the best way to check to see if there is data in a directory before deleting it? I am browsing through a couple pages to find some pics using wget and of course every page does not have an image on it but the directory is still created.

dir = 'Files\\%s' % (directory)
os.mkdir(dir)
cmd = 'wget -r -l1 -nd -np -A.jpg,.png,.gif -P %s %s' %(dir,  i[1])
os.system(cmd)
if not os.path.isdir(dir):
    os.rmdir(dir)

I would like to test to see if a file was dropped in the directory after it was created. If nothing is there...delete it.

Thanks,
Adam

Body:
I will go with EAFP like so:

try:
    os.rmdir(dir)
except OSError as ex:
    if ex.errno == errno.ENOTEMPTY:
        print "directory not empty"

os.rmdir will not delete a directory that is not empty.


问:
SAS-Append data to a table and increment a flag on the resultant table
I have two tables on SAS "Table_A" and "Table_A_Archive", as part my ETL process "Table_A" is created on a daily basis and the data should be archived on "Table_A_Archive". When the data is archived on "Table_A_Archive" a a flag will be created/updated "DT_FLAG".

On the first day this is how the table would look like

```<code> "Table_A"
 | ID  | Load_Date
 ------ -------------
 | 100 | 01JUN2020:12:13:56

 "Table_A_Archive"
  | ID  | Load_Date           | DT_FLAG
  ------ --------------------- ---------
  | 100 | 01JUN2020:12:13:56  | 1
</code>```

On Day 2

```<code> "Table_A"
 | ID  | Load_Date
 ------ ------------
 | 101 | 02JUN2020:12:13:56

 "Table_A_Archive"
 | ID  | Load_Date           | DT_FLAG
 ------ --------------------- ---------
 | 100 | 01JUN2020:12:13:56  | 2
 | 101 | 02JUN2020:12:13:56  | 1
</code>```

The new data should be loaded with the DT_FLAG of 1 and old records DT_FLAG should be incremented by 1. Load_Date is the pivotal point. I have written a SAS code but it seems a bit messy, can someone please help me with a SAS Datastep

```<code> %macro Cntl_archive(table_name=,arch_table_name=);
 %GLOBAL WRK;

 %if %sysfunc(exist(&arch_table_name.))  %then %do;

 proc append base=&arch_table_name. data=&table_name. force;
 run;

 proc sql;
 Create table TEMP as
 Select distinct Load_Date,Load_Date as WRK from &arch_table_name.
 order by Load_Date desc
 ;quit;

 proc rank data=TEMP descending out=TEMP;
 var WRK;
 ranks count;
 run;

 data &arch_table_name. (drop=DT_FLAG);
 set    &arch_table_name.;
 run;

 proc sql;
 Create table &arch_table_name. as
 Select T0.*,T1.count as DT_FLAG from &arch_table_name. T0
 inner join TEMP T1 on T0.Load_Date=T1.Load_Date
 ;quit
 %end;
 %else %do;

data &arch_table_name.;
set &table_name.;
DT_FLAG= 1;
IS_ACTIVE='';
run;
%end;
%mend Cntl_archive;
</code>```

答:
Consider again proc sql with a count correlated subquery. Unfortunately, SAS does not allow updating a table with values to itself, so a temp table copy is used. Below assumes ID is incremented with each day.

```<code>proc sql;
    insert into Table_A_Archive (ID, Load_Date)
    select ID, Load_Date
    from Table_A;

    create table temp as
    select ID, Load_Date from Table_A_Archive;

    update Table_A_Archive t
    set DT_Flag = (select count(*)
                   from temp sub
                   where t.ID <= sub.ID
                   and   t.Load_Date = sub.Load_Date);

    drop table temp;
quit;
</code>```

Body:
I have two tables on SAS "Table_A" and "Table_A_Archive", as part my ETL process "Table_A" is created on a daily basis and the data should be archived on "Table_A_Archive". When the data is archived on "Table_A_Archive" a a flag will be created/updated "DT_FLAG".

On the first day this is how the table would look like

 "Table_A"
 | ID  | Load_Date
 ------ -------------
 | 100 | 01JUN2020:12:13:56

 "Table_A_Archive"
  | ID  | Load_Date           | DT_FLAG
  ------ --------------------- ---------
  | 100 | 01JUN2020:12:13:56  | 1

On Day 2

 "Table_A"
 | ID  | Load_Date
 ------ ------------
 | 101 | 02JUN2020:12:13:56

 "Table_A_Archive"
 | ID  | Load_Date           | DT_FLAG
 ------ --------------------- ---------
 | 100 | 01JUN2020:12:13:56  | 2
 | 101 | 02JUN2020:12:13:56  | 1

The new data should be loaded with the DT_FLAG of 1 and old records DT_FLAG should be incremented by 1. Load_Date is the pivotal point. I have written a SAS code but it seems a bit messy, can someone please help me with a SAS Datastep

 %macro Cntl_archive(table_name=,arch_table_name=);
 %GLOBAL WRK;

 %if %sysfunc(exist(&arch_table_name.))  %then %do;

 proc append base=&arch_table_name. data=&table_name. force;
 run;

 proc sql;
 Create table TEMP as
 Select distinct Load_Date,Load_Date as WRK from &arch_table_name.
 order by Load_Date desc
 ;quit;

 proc rank data=TEMP descending out=TEMP;
 var WRK;
 ranks count;
 run;

 data &arch_table_name. (drop=DT_FLAG);
 set    &arch_table_name.;
 run;

 proc sql;
 Create table &arch_table_name. as
 Select T0.*,T1.count as DT_FLAG from &arch_table_name. T0
 inner join TEMP T1 on T0.Load_Date=T1.Load_Date
 ;quit
 %end;
 %else %do;

data &arch_table_name.;
set &table_name.;
DT_FLAG= 1;
IS_ACTIVE='';
run;
%end;
%mend Cntl_archive;

Body:
Consider again proc sql with a count correlated subquery. Unfortunately, SAS does not allow updating a table with values to itself, so a temp table copy is used. Below assumes ID is incremented with each day.

proc sql;
    insert into Table_A_Archive (ID, Load_Date)
    select ID, Load_Date
    from Table_A;

    create table temp as
    select ID, Load_Date from Table_A_Archive;

    update Table_A_Archive t
    set DT_Flag = (select count(*)
                   from temp sub
                   where t.ID <= sub.ID
                   and   t.Load_Date = sub.Load_Date);

    drop table temp;
quit;


问:
How to add a search bar functionality to all the templates of django?
I am trying to add a search bar in my django site. I added a form in my base template. I have added the functionality to my home view. It is working in the home view. But not in other views. I tried creating a new view but i.e. Is not working too. Please tell me what should I do?

views.py:

def search(request):
    if request.method == "GET":
        query = request.GET.get('q').split()
        if query:
            queryset =  reduce(
                operator.__or__,
                [
                    Q(title__icontains=queries) | Q(content__icontains=queries)
                    for queries in query
                ]
            )
            posts = Post.objects.filter(queryset).distinct()

            return render(request, "blog/home.html", {'posts': posts})
```

<p>template:-</p>

```<form class="form-inline my-2 my-lg-0">
                            <input class="form-control mr-sm-2" action="{% url 'blog-home' %}" method="GET" type="search" placeholder="Search Posts" aria-label="Search" name="q">
                            <button class="btn btn-outline-success my-2 my-sm-0" type="submit">Search</button>
                        </form>
```

<p>urls.py</p>

```urlpatterns = [
    path('', PostListView.as_view(), name='blog-home'),
    path('user/<str:username>', UserPostListView.as_view(), name='user-posts'),
    path('post/<int:pk>/', PostDetailView.as_view(), name='post-detail'),
    path('post/new/', PostCreateView.as_view(), name='post-create'),
    path('post/<int:pk>/update/', PostUpdateView.as_view(), name='post-update'),
    path('post/<int:pk>/delete/', PostDeleteView.as_view(), name='post-delete'),
    path('about/', views.about, name='blog-about'),
    path('comment/<int:pk>/approve/', views.comment_approve, name='comment_approve'),
    path('comment/<int:pk>/remove/', views.comment_remove, name='comment_remove'),
    path('comment/<int:pk>/delete/', views.comment_delete, name='comment_delete'),
    path('summernote/', include('django_summernote.urls')),
    path('feedback/', views.contact_us, name='contact_us'),
    path('search/', views.search),
]
```

答:
In your template form tag add the URL name where you want to load the search function. This way you can search from any page and will redirect you that page. If you don't get it I can provide you with a reference code so that you can grasp some idea. </p>

```<code><form method="GET" action="{% url 'courses' %}">
</code>```

Body:
I am trying to add a search bar in my django site. I added a form in my base template. I have added the functionality to my home view. It is working in the home view. But not in other views. I tried creating a new view but i.e. Is not working too. Please tell me what should I do?

views.py:

def search(request):
    if request.method == "GET":
        query = request.GET.get('q').split()
        if query:
            queryset =  reduce(
                operator.__or__,
                [
                    Q(title__icontains=queries) | Q(content__icontains=queries)
                    for queries in query
                ]
            )
            posts = Post.objects.filter(queryset).distinct()

            return render(request, "blog/home.html", {'posts': posts})

template:-

<form class="form-inline my-2 my-lg-0">
                            <input class="form-control mr-sm-2" action="{% url 'blog-home' %}" method="GET" type="search" placeholder="Search Posts" aria-label="Search" name="q">
                            <button class="btn btn-outline-success my-2 my-sm-0" type="submit">Search</button>
                        </form>

urls.py

urlpatterns = [
    path('', PostListView.as_view(), name='blog-home'),
    path('user/<str:username>', UserPostListView.as_view(), name='user-posts'),
    path('post/<int:pk>/', PostDetailView.as_view(), name='post-detail'),
    path('post/new/', PostCreateView.as_view(), name='post-create'),
    path('post/<int:pk>/update/', PostUpdateView.as_view(), name='post-update'),
    path('post/<int:pk>/delete/', PostDeleteView.as_view(), name='post-delete'),
    path('about/', views.about, name='blog-about'),
    path('comment/<int:pk>/approve/', views.comment_approve, name='comment_approve'),
    path('comment/<int:pk>/remove/', views.comment_remove, name='comment_remove'),
    path('comment/<int:pk>/delete/', views.comment_delete, name='comment_delete'),
    path('summernote/', include('django_summernote.urls')),
    path('feedback/', views.contact_us, name='contact_us'),
    path('search/', views.search),
]

Body:
In your template form tag add the URL name where you want to load the search function. This way you can search from any page and will redirect you that page. If you don't get it I can provide you with a reference code so that you can grasp some idea.

<form method="GET" action="{% url 'courses' %}">


问:
How to comprehend that the type of `std::function` depends solely on its call signature?
As per the documentation(http://www.cplusplus.com/reference/functional/function/), which says that:

Class that can wrap any kind of callable element (such as functions
  and function objects) into a copyable object, and whose type depends
  solely on its call signature (and not on the callable element type
  itself).

How to comprehend that the type of std::function depends solely on its call signature(and not on the callable element type itself)? Could somebody make it clear by giving some simple examples?I would be greatful to have some help with this question.

答:
The template type effectively denotes a function signature (excluding name of course).

std::function<bool(Bar const&, Foo const&)>

Can hold a functor, member function pointer, function pointer or lambda. But the callable must have the bool (Bar const&, Foo const&) signature.

```<code>class Foo {};
class Bar {};
class FunctorEx
{
  public:
    bool operator()(Bar const&, Foo const&)
    {
        return true;
    }
} FunctorExInst;

class MemFunction
{
  public:
    bool MemFunctionEx(Bar const&, Foo const&)
    {
        return true;
    }
} MemFunctionInst;

bool FunctionEx(Bar const&, Foo const&)
{
    return true;
}

int main()
{
    auto LambdaEx = [] (Bar const&, Foo const&) -> bool
    {
        return true;
    };

    std::function<bool(Bar const&, Foo const&)> exFunctionVar;
    exFunctionVar = std::bind(&MemFunction::MemFunctionEx, &MemFunctionInst, std::placeholders::_1, std::placeholders::_2);
    exFunctionVar = FunctorExInst;
    exFunctionVar = FunctionEx;
    exFunctionVar = LambdaEx;
}
</code>```

Despite MemFunctionInst, FunctorExInst, FunctionEx, LambdaEx all being of different types, they can all be assigned to the same std::function variable due to a technique called type erasure.

Body:
The template type effectively denotes a function signature (excluding name of course).

std::function<bool(Bar const&, Foo const&)>

Can hold a functor, member function pointer, function pointer or lambda. But the callable must have the bool (Bar const&, Foo const&) signature.

class Foo {};
class Bar {};
class FunctorEx
{
  public:
    bool operator()(Bar const&, Foo const&)
    {
        return true;
    }
} FunctorExInst;

class MemFunction
{
  public:
    bool MemFunctionEx(Bar const&, Foo const&)
    {
        return true;
    }
} MemFunctionInst;

bool FunctionEx(Bar const&, Foo const&)
{
    return true;
}

int main()
{
    auto LambdaEx = [] (Bar const&, Foo const&) -> bool
    {
        return true;
    };

    std::function<bool(Bar const&, Foo const&)> exFunctionVar;
    exFunctionVar = std::bind(&MemFunction::MemFunctionEx, &MemFunctionInst, std::placeholders::_1, std::placeholders::_2);
    exFunctionVar = FunctorExInst;
    exFunctionVar = FunctionEx;
    exFunctionVar = LambdaEx;
}

Despite MemFunctionInst, FunctorExInst, FunctionEx, LambdaEx all being of different types, they can all be assigned to the same std::function variable due to a technique called type erasure.


问:
How to save and search for EXACT date?

Setup

I'm creating a react app to save and retrieve daily reports from MongoDb Atlas database. There can be only one report per day per user. The search criteria for daily reports is the date of course.

Logic and code

The app has a calendar widget from material-ui datepicker. It returns the selected date with the current UTC time. But if I search the database using that directly, then obviously I won't find anything because of the difference of hours:minutes:seconds:milliseconds. It's impossible that the Date() that you're searching for would have the exact millisecond match with the time the report was created. So my idea was to store the date for each report with a time of exactly 12am i.e. 00:00:00:000Z as well as search the database with a date of time 12am.

```<code>selectedDate.setUTCHours(0,0,0,0); //sets the time to 12am
selectedDate.toISOString();
</code>```

Question

Is this the right approach for this case? Will I face problems down the line doing it this way? How should the date be stored if it's the search criteria and if it has to be the exact match? Also it must serve any user around the world to save and retrieve their reports.

Thanks,
Suraj

答:
You can store the date as you like, just in your db query use between condition</p>

```<code>SELECT * FROM reports WHERE date BETWEEN '2020-05-01 00:00:00' AND '2020-05-01 23:59:59'
</code>```

Title: How to save and search for EXACT date?

Body:
Setup

I'm creating a react app to save and retrieve daily reports from MongoDb Atlas database. There can be only one report per day per user. The search criteria for daily reports is the date of course.

Logic and code

The app has a calendar widget from material-ui datepicker. It returns the selected date with the current UTC time. But if I search the database using that directly, then obviously I won't find anything because of the difference of hours:minutes:seconds:milliseconds. It's impossible that the Date() that you're searching for would have the exact millisecond match with the time the report was created. So my idea was to store the date for each report with a time of exactly 12am i.e. 00:00:00:000Z as well as search the database with a date of time 12am.

selectedDate.setUTCHours(0,0,0,0); //sets the time to 12am
selectedDate.toISOString();

Question

Is this the right approach for this case? Will I face problems down the line doing it this way? How should the date be stored if it's the search criteria and if it has to be the exact match? Also it must serve any user around the world to save and retrieve their reports.

Thanks,
Suraj

Body:
You can store the date as you like, just in your db query use between condition

SELECT * FROM reports WHERE date BETWEEN '2020-05-01 00:00:00' AND '2020-05-01 23:59:59'


问:
In subversion I want to Revert back to a revison then step through each revision to see where it broke
My project broke somewhere in the past 25 revisions. I need to isolate the file causing the problem, and would like to find the revision where the problem was introduced.

I was thinking of using svn update  -r 404 and then each one after that 405,406 ... etc

Is there a better way?

答:
svn-bisect also allows you to automate the search if you can supply a command for the bug you're looking for:

$ svn-bisect run 'command [arg ...]'

To speed up the search I would recommend to clone the svn repository into a local git or hg repository, and run the bisect from there.  Both support bisect automation:

$ hg bisect -c 'cmd ...'

or

$ git bisect run '...'


问:
Can't access my laptop's localhost through an Android app
So I did a research before posting this and the solutions I found didn't work, more precisely:

-Connecting to my laptop's IPv4 192.168.XXX.XXX - didn't work

-Conecting to 10.0.2.2 (plus the port) - didn't work

I need to test an API i built using Django Rest framework so I can get the json it returns, but i cant access through an Android app i'm building (I'm testing with a real device, not an emulator). Internet permissions are set on Manifest and i can access remote websites normally. Just can't reach my laptop's localhost(they are in the same network)

I'm pretty new to Android and Python and Django as well (used to built Django's Rest Framework API).

EDIT: I use localhost:8000/snippets.json or smth like this to connect on my laptop.

PS: I read something about XAMP server... do I need it in this case?

Thanks in advance

答:
Others have already given the answer but those solutions didn't work for me when I was running Django server on Ubuntu and was trying to access it from my Mobile app.

Following is what worked for me:

Step 1: Run server to access it from your IP

```<code>python manage.py runserver 0.0.0.0:8000
</code>```

Step 2: Open port in firewall (Missing step in above given answers - Needed for Ubuntu)

```<code>$ sudo ufw enable
$ sudo ufw allow 8000
</code>```

Body:
Others have already given the answer but those solutions didn't work for me when I was running Django server on Ubuntu and was trying to access it from my Mobile app.

Following is what worked for me:

Step 1: Run server to access it from your IP

python manage.py runserver 0.0.0.0:8000

Step 2: Open port in firewall (Missing step in above given answers - Needed for Ubuntu)

$ sudo ufw enable
$ sudo ufw allow 8000