问:
netty - ssl/tls two way authentication
I already tried Two way SSL authentication in Netty

but the example is not showing any info anymore, just a 404 Not Found. I have found some help here:

https://github.com/code4craft/netty-learning/blob/master/netty-3.7/src/main/java/org/jboss/netty/example/securechat/SecureChatSslContextFactory.java

but my code still fails after doing what is told allthough my code does not use the exact same api-calls as the sslchat (order of managers added to init).

Here is my code and the error. The error does not indicate which ssl/tls error it is. I have checked both certificates/private keys (for both sides of the client/server) using just one way auth and encryption and it works.

Server side code:

```<code>    KeyStore serverKeyStore = handleServerKeystore(sipListener);
KeyManagerFactory serverKmf = KeyManagerFactory
                                            .getInstance(TrustManagerFactory.getDefaultAlgorithm());
KeyStore clientKeyStore = KeyStore.getInstance("JKS");
clientKeyStore.load(null, SipListener.KEYSTORE_PASSWORD.toCharArray());
for (ClientCertificate clientCertCert : sipListener.getClientCertificates()) {
Certificate clientCert = CertificateFactory.getInstance("X.509")
                                                .generateCertificate(new ByteArrayInputStream(
                                                        clientCertCert.getClientCertificate().getBytes()));
                                        clientKeyStore.setCertificateEntry(clientCertCert.getClientCertificateAlias(),
                                                clientCert);
                                    }

KeyStore.Builder serverBuilder = KeyStore.Builder.newInstance(serverKeyStore,
                                            new KeyStore.PasswordProtection(SipListener.KEYSTORE_PASSWORD.toCharArray()));
KeyStore.Builder clientBuilder = KeyStore.Builder.newInstance(clientKeyStore,
                                            new KeyStore.PasswordProtection(SipListener.KEYSTORE_PASSWORD.toCharArray()));
                                    serverKmf.init(new KeyStoreBuilderParameters(
                                            Arrays.asList(new KeyStore.Builder[] { serverBuilder, clientBuilder })));

// Initialize the SSLContext to work with our key managers.
SslContext serverContext = SslContextBuilder.forServer(serverKmf).build();

SSLEngine sslEngine = serverContext.newEngine(ch.alloc());
SSLParameters params = new SSLParameters();
List<SNIMatcher> matchers = new LinkedList<>();
SNIMatcher matcher = new SNIMatcher(0) {

                                        @Override
                                        public boolean matches(SNIServerName serverName) {
                                            return true;
                                        }
                                    };
matchers.add(matcher);
params.setSNIMatchers(matchers);
sslEngine.setSSLParameters(params);
//sslEngine.setUseClientMode(false);
sslEngine.setNeedClientAuth(true);
cp.addFirst("ssl", new SslHandler(sslEngine));
//More codecs follow
</code>```

Client side code:

```<code>SslContextBuilder sslBuilder = SslContextBuilder.forClient();
SslContext cont2 = null;
KeyManagerFactory keyManagerFactory = KeyManagerFactory
                                .getInstance(KeyManagerFactory.getDefaultAlgorithm());
KeyStore serverKeyStore = KeyStore.getInstance("BKS");
serverKeyStore.load(
                                new ByteArrayInputStream(
                                        decoder.decode(sipSettingsBean.getSipSettingsServerBeans().get(0).getKeystoreB64().getBytes())),
SipListener.KEYSTORE_PASSWORD.toCharArray());
if (!serverKeyStore.isKeyEntry(sipSettingsBean.getSipSettingsServerBeans().get(0).getKeystoreAlias()))
                            throw new IllegalArgumentException(
                                    "Server Keystore file has no matching key for given alias.");
keyManagerFactory.init(serverKeyStore,
                                SipListener.KEYSTORE_PASSWORD.toCharArray());
sslBuilder.keyManager(keyManagerFactory);

TrustManagerFactory trustManagerFactory = TrustManagerFactory
                                .getInstance(TrustManagerFactory.getDefaultAlgorithm());
// truststore
KeyStore clientKeyStore = KeyStore.getInstance("BKS");
clientKeyStore.load(null, SipListener.KEYSTORE_PASSWORD.toCharArray());
for (Cert clientCertCert : sipSettingsBean.getSipSettingsServerBeans().get(0).getCerts()) {
Certificate clientCert = CertificateFactory.getInstance("X.509")
                                    .generateCertificate(new ByteArrayInputStream(
                                            clientCertCert.getCert().getBytes()));
clientKeyStore.setCertificateEntry(
                                    clientCertCert.getAlias(), clientCert);
if (!clientKeyStore.isCertificateEntry(clientCertCert.getAlias()))
                                throw new IllegalArgumentException(
                                        "Client Keystore file has no matching key for given alias.");
                        }

trustManagerFactory.init(clientKeyStore);
sslBuilder.trustManager(trustManagerFactory);

cont2 = sslBuilder.build();
SSLEngine engine = cont2.newEngine(ch.alloc(), toHostname,
                                portDestination);
engine.setEnabledProtocols(new String[]{"TLSv1.2"});
SSLParameters params = new SSLParameters();
List<SNIMatcher> matchers = new LinkedList<>();
SNIMatcher matcher = new SNIMatcher(0) {

                            @Override
                            public boolean matches(SNIServerName serverName) {
                              return true;
                            }
                        };
matchers.add(matcher);
params.setSNIMatchers(matchers);
engine.setSSLParameters(params);
ch.pipeline().addLast("ssl", new SslHandler(engine, false));
//More codecs follow
</code>```

Error on client side:

```<code>2019-01-26 18:18:54.784 31491-31672/xx.xxxxxxxxxx.xxxxxxxxx D/no.tobiassenit.sipclient.network.RegisterAttemptSSL: request sent
2019-01-26 18:18:54.786 31491-31672/xx.xxxxxxxxxx.xxxxxxxxx D/no.tobiassenit.sipclient.network.RegisterAttemptSSL: request did not time out
2019-01-26 18:18:54.789 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err: io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Read error: ssl=0x774f846f08: Failure in SSL library, usually a protocol error
2019-01-26 18:18:54.789 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err: error:10000412:SSL routines:OPENSSL_internal:SSLV3_ALERT_BAD_CERTIFICATE (external/boringssl/src/ssl/tls_record.cc:592 0x774f890d08:0x00000001)
2019-01-26 18:18:54.789 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
2019-01-26 18:18:54.791 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:646)
2019-01-26 18:18:54.791 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:581)
2019-01-26 18:18:54.791 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498)
2019-01-26 18:18:54.791 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:460)
2019-01-26 18:18:54.791 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
2019-01-26 18:18:54.791 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
2019-01-26 18:18:54.791 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at java.lang.Thread.run(Thread.java:764)
2019-01-26 18:18:54.792 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err: Caused by: javax.net.ssl.SSLHandshakeException: Read error: ssl=0x774f846f08: Failure in SSL library, usually a protocol error
2019-01-26 18:18:54.792 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err: error:10000412:SSL routines:OPENSSL_internal:SSLV3_ALERT_BAD_CERTIFICATE (external/boringssl/src/ssl/tls_record.cc:592 0x774f890d08:0x00000001)
2019-01-26 18:18:54.792 31491-31771/v W/System.err:     at com.android.org.conscrypt.SSLUtils.toSSLHandshakeException(SSLUtils.java:331)
2019-01-26 18:18:54.792 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.ConscryptEngine.convertException(ConscryptEngine.java:1138)
2019-01-26 18:18:54.792 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:893)
2019-01-26 18:18:54.792 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:713)
2019-01-26 18:18:54.792 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:678)
2019-01-26 18:18:54.793 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.Java8EngineWrapper.unwrap(Java8EngineWrapper.java:236)
2019-01-26 18:18:54.793 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:294)
2019-01-26 18:18:54.793 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1275)
2019-01-26 18:18:54.793 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1177)
2019-01-26 18:18:54.793 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1221)
2019-01-26 18:18:54.793 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
2019-01-26 18:18:54.793 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
2019-01-26 18:18:54.793 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:   ... 16 more
2019-01-26 18:18:54.794 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err: Caused by: javax.net.ssl.SSLProtocolException: Read error: ssl=0x774f846f08: Failure in SSL library, usually a protocol error
2019-01-26 18:18:54.794 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err: error:10000412:SSL routines:OPENSSL_internal:SSLV3_ALERT_BAD_CERTIFICATE (external/boringssl/src/ssl/tls_record.cc:592 0x774f890d08:0x00000001)
2019-01-26 18:18:54.794 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.NativeCrypto.ENGINE_SSL_read_direct(Native Method)
2019-01-26 18:18:54.794 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.NativeSsl.readDirectByteBuffer(NativeSsl.java:521)
2019-01-26 18:18:54.794 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.ConscryptEngine.readPlaintextDataDirect(ConscryptEngine.java:1099)
2019-01-26 18:18:54.794 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.ConscryptEngine.readPlaintextDataHeap(ConscryptEngine.java:1119)
2019-01-26 18:18:54.795 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1091)
2019-01-26 18:18:54.795 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:841)
2019-01-26 18:18:54.795 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:   ... 25 more
</code>```

Error on server side:

```<code> io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: null cert chain
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:646)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:581)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:460)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:748)Caused by: javax.net.ssl.SSLHandshakeException: null cert chain
    at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1441)
    at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535)
    at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:813)
    at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781)
    at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
    at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:294)
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1275)
    at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1177)
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1221)
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
    ... 16 moreCaused by: javax.net.ssl.SSLHandshakeException: null cert chain
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:201)
    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1672)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:309)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:297)
    at sun.security.ssl.ServerHandshaker.clientCertificate(ServerHandshaker.java:1942)
    at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:236)
    at sun.security.ssl.Handshaker.processLoop(Handshaker.java:984)
    at sun.security.ssl.Handshaker$1.run(Handshaker.java:924)
    at sun.security.ssl.Handshaker$1.run(Handshaker.java:921)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1379)
    at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1435)
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1343)
    ... 20 more
</code>```

|

And of course: netty: 4.1.28.Final, Win10 and java version "1.8.0_151"

I tried netty 4.1.33.Final as well. Same result.

As you may have noticed I have ignored the test for hostname in the engine. This might be just for the server handshake and not for the client (private key) handshake. Please advice if you can on how to ignore the client hostname part of the handshake as well unless it is already done (I have tried doing it on both sides) as my application will not be able to support a dns name for the client. (It will be able to support it for the server but for the moment ignoring it).

I tried netty 4.1.34.Final-20190208.192045-20. Same result.
I tried netty 4.1.34.Final. Same result.
I tried netty 4.1.35.Final-SNAPSHOT. Same result.

答:
I was facing this problem while performing two way SSL with Netty. I solved it by using following configuration:

```<code>public HttpClient getHttpClient(HttpClientProperties properties){

        // configure pool resources
        HttpClientProperties.Pool pool = properties.getPool();

        ConnectionProvider connectionProvider;
        if (pool.getType() == DISABLED) {
            connectionProvider = ConnectionProvider.newConnection();
        }
        else if (pool.getType() == FIXED) {
            connectionProvider = ConnectionProvider.fixed(pool.getName(),
                    pool.getMaxConnections(), pool.getAcquireTimeout());
        }
        else {
            connectionProvider = ConnectionProvider.elastic(pool.getName());
        }

        HttpClient httpClient = HttpClient.create(connectionProvider)
                .tcpConfiguration(tcpClient -> {

                    if (properties.getConnectTimeout() != null) {
                        tcpClient = tcpClient.option(
                                ChannelOption.CONNECT_TIMEOUT_MILLIS,
                                properties.getConnectTimeout());
                    }

                    // configure proxy if proxy host is set.
                    HttpClientProperties.Proxy proxy = properties.getProxy();

                    if (StringUtils.hasText(proxy.getHost())) {

                        tcpClient = tcpClient.proxy(proxySpec -> {
                            ProxyProvider.Builder builder = proxySpec
                                    .type(ProxyProvider.Proxy.HTTP)
                                    .host(proxy.getHost());

                            PropertyMapper map = PropertyMapper.get();

                            map.from(proxy::getPort).whenNonNull().to(builder::port);
                            map.from(proxy::getUsername).whenHasText()
                                    .to(builder::username);
                            map.from(proxy::getPassword).whenHasText()
                                    .to(password -> builder.password(s -> password));
                            map.from(proxy::getNonProxyHostsPattern).whenHasText()
                                    .to(builder::nonProxyHosts);
                        });
                    }
                    return tcpClient;
                });

        HttpClientProperties.Ssl ssl = properties.getSsl();
        if (ssl.getTrustedX509CertificatesForTrustManager().length > 0
                || ssl.isUseInsecureTrustManager()) {
            httpClient = httpClient.secure(sslContextSpec -> {
                // configure ssl
                SslContextBuilder sslContextBuilder = SslContextBuilder.forClient();

                X509Certificate[] trustedX509Certificates = ssl
                        .getTrustedX509CertificatesForTrustManager();
                if (trustedX509Certificates.length > 0) {
                    sslContextBuilder.trustManager(trustedX509Certificates);
                }
                else if (ssl.isUseInsecureTrustManager()) {
                    sslContextBuilder
                            .trustManager(InsecureTrustManagerFactory.INSTANCE);
                }

                sslContextSpec.sslContext(sslContextBuilder)
                        .defaultConfiguration(ssl.getDefaultConfigurationType())
                        .handshakeTimeout(ssl.getHandshakeTimeout())
                        .closeNotifyFlushTimeout(ssl.getCloseNotifyFlushTimeout())
                        .closeNotifyReadTimeout(ssl.getCloseNotifyReadTimeout())
                        .handlerConfigurator(
                                (handler)->{
                                    SSLEngine engine = handler.engine();
                                    //engine.setNeedClientAuth(true);
                                    SSLParameters params = new SSLParameters();
                                    List<SNIMatcher> matchers = new LinkedList<>();
                                    SNIMatcher matcher = new SNIMatcher(0) {

                                        @Override
                                        public boolean matches(SNIServerName serverName) {
                                            return true;
                                        }
                                    };
                                    matchers.add(matcher);
                                    params.setSNIMatchers(matchers);
                                    engine.setSSLParameters(params);
                                }
                        )
                ;
            });
        }

        return httpClient;

    }
</code>```

The above code snippet uses handlerConfigurator provided by netty's Builder present inside the SslProvider class to customize the SslHandler to avoid hostname matching.

Title: netty - ssl/tls two way authentication

Body:
I already tried Two way SSL authentication in Netty

but the example is not showing any info anymore, just a 404 Not Found. I have found some help here:

https://github.com/code4craft/netty-learning/blob/master/netty-3.7/src/main/java/org/jboss/netty/example/securechat/SecureChatSslContextFactory.java

but my code still fails after doing what is told allthough my code does not use the exact same api-calls as the sslchat (order of managers added to init).

Here is my code and the error. The error does not indicate which ssl/tls error it is. I have checked both certificates/private keys (for both sides of the client/server) using just one way auth and encryption and it works.

Server side code:

    KeyStore serverKeyStore = handleServerKeystore(sipListener);
KeyManagerFactory serverKmf = KeyManagerFactory
                                            .getInstance(TrustManagerFactory.getDefaultAlgorithm());
KeyStore clientKeyStore = KeyStore.getInstance("JKS");
clientKeyStore.load(null, SipListener.KEYSTORE_PASSWORD.toCharArray());
for (ClientCertificate clientCertCert : sipListener.getClientCertificates()) {
Certificate clientCert = CertificateFactory.getInstance("X.509")
                                                .generateCertificate(new ByteArrayInputStream(
                                                        clientCertCert.getClientCertificate().getBytes()));
                                        clientKeyStore.setCertificateEntry(clientCertCert.getClientCertificateAlias(),
                                                clientCert);
                                    }

KeyStore.Builder serverBuilder = KeyStore.Builder.newInstance(serverKeyStore,
                                            new KeyStore.PasswordProtection(SipListener.KEYSTORE_PASSWORD.toCharArray()));
KeyStore.Builder clientBuilder = KeyStore.Builder.newInstance(clientKeyStore,
                                            new KeyStore.PasswordProtection(SipListener.KEYSTORE_PASSWORD.toCharArray()));
                                    serverKmf.init(new KeyStoreBuilderParameters(
                                            Arrays.asList(new KeyStore.Builder[] { serverBuilder, clientBuilder })));

// Initialize the SSLContext to work with our key managers.
SslContext serverContext = SslContextBuilder.forServer(serverKmf).build();

SSLEngine sslEngine = serverContext.newEngine(ch.alloc());
SSLParameters params = new SSLParameters();
List<SNIMatcher> matchers = new LinkedList<>();
SNIMatcher matcher = new SNIMatcher(0) {

                                        @Override
                                        public boolean matches(SNIServerName serverName) {
                                            return true;
                                        }
                                    };
matchers.add(matcher);
params.setSNIMatchers(matchers);
sslEngine.setSSLParameters(params);
//sslEngine.setUseClientMode(false);
sslEngine.setNeedClientAuth(true);
cp.addFirst("ssl", new SslHandler(sslEngine));
//More codecs follow

Client side code:

SslContextBuilder sslBuilder = SslContextBuilder.forClient();
SslContext cont2 = null;
KeyManagerFactory keyManagerFactory = KeyManagerFactory
                                .getInstance(KeyManagerFactory.getDefaultAlgorithm());
KeyStore serverKeyStore = KeyStore.getInstance("BKS");
serverKeyStore.load(
                                new ByteArrayInputStream(
                                        decoder.decode(sipSettingsBean.getSipSettingsServerBeans().get(0).getKeystoreB64().getBytes())),
SipListener.KEYSTORE_PASSWORD.toCharArray());
if (!serverKeyStore.isKeyEntry(sipSettingsBean.getSipSettingsServerBeans().get(0).getKeystoreAlias()))
                            throw new IllegalArgumentException(
                                    "Server Keystore file has no matching key for given alias.");
keyManagerFactory.init(serverKeyStore,
                                SipListener.KEYSTORE_PASSWORD.toCharArray());
sslBuilder.keyManager(keyManagerFactory);

TrustManagerFactory trustManagerFactory = TrustManagerFactory
                                .getInstance(TrustManagerFactory.getDefaultAlgorithm());
// truststore
KeyStore clientKeyStore = KeyStore.getInstance("BKS");
clientKeyStore.load(null, SipListener.KEYSTORE_PASSWORD.toCharArray());
for (Cert clientCertCert : sipSettingsBean.getSipSettingsServerBeans().get(0).getCerts()) {
Certificate clientCert = CertificateFactory.getInstance("X.509")
                                    .generateCertificate(new ByteArrayInputStream(
                                            clientCertCert.getCert().getBytes()));
clientKeyStore.setCertificateEntry(
                                    clientCertCert.getAlias(), clientCert);
if (!clientKeyStore.isCertificateEntry(clientCertCert.getAlias()))
                                throw new IllegalArgumentException(
                                        "Client Keystore file has no matching key for given alias.");
                        }

trustManagerFactory.init(clientKeyStore);
sslBuilder.trustManager(trustManagerFactory);

cont2 = sslBuilder.build();
SSLEngine engine = cont2.newEngine(ch.alloc(), toHostname,
                                portDestination);
engine.setEnabledProtocols(new String[]{"TLSv1.2"});
SSLParameters params = new SSLParameters();
List<SNIMatcher> matchers = new LinkedList<>();
SNIMatcher matcher = new SNIMatcher(0) {

                            @Override
                            public boolean matches(SNIServerName serverName) {
                              return true;
                            }
                        };
matchers.add(matcher);
params.setSNIMatchers(matchers);
engine.setSSLParameters(params);
ch.pipeline().addLast("ssl", new SslHandler(engine, false));
//More codecs follow

Error on client side:

2019-01-26 18:18:54.784 31491-31672/xx.xxxxxxxxxx.xxxxxxxxx D/no.tobiassenit.sipclient.network.RegisterAttemptSSL: request sent
2019-01-26 18:18:54.786 31491-31672/xx.xxxxxxxxxx.xxxxxxxxx D/no.tobiassenit.sipclient.network.RegisterAttemptSSL: request did not time out
2019-01-26 18:18:54.789 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err: io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Read error: ssl=0x774f846f08: Failure in SSL library, usually a protocol error
2019-01-26 18:18:54.789 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err: error:10000412:SSL routines:OPENSSL_internal:SSLV3_ALERT_BAD_CERTIFICATE (external/boringssl/src/ssl/tls_record.cc:592 0x774f890d08:0x00000001)
2019-01-26 18:18:54.789 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
2019-01-26 18:18:54.790 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
2019-01-26 18:18:54.791 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:646)
2019-01-26 18:18:54.791 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:581)
2019-01-26 18:18:54.791 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498)
2019-01-26 18:18:54.791 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:460)
2019-01-26 18:18:54.791 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
2019-01-26 18:18:54.791 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
2019-01-26 18:18:54.791 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at java.lang.Thread.run(Thread.java:764)
2019-01-26 18:18:54.792 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err: Caused by: javax.net.ssl.SSLHandshakeException: Read error: ssl=0x774f846f08: Failure in SSL library, usually a protocol error
2019-01-26 18:18:54.792 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err: error:10000412:SSL routines:OPENSSL_internal:SSLV3_ALERT_BAD_CERTIFICATE (external/boringssl/src/ssl/tls_record.cc:592 0x774f890d08:0x00000001)
2019-01-26 18:18:54.792 31491-31771/v W/System.err:     at com.android.org.conscrypt.SSLUtils.toSSLHandshakeException(SSLUtils.java:331)
2019-01-26 18:18:54.792 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.ConscryptEngine.convertException(ConscryptEngine.java:1138)
2019-01-26 18:18:54.792 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:893)
2019-01-26 18:18:54.792 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:713)
2019-01-26 18:18:54.792 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:678)
2019-01-26 18:18:54.793 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.Java8EngineWrapper.unwrap(Java8EngineWrapper.java:236)
2019-01-26 18:18:54.793 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:294)
2019-01-26 18:18:54.793 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1275)
2019-01-26 18:18:54.793 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1177)
2019-01-26 18:18:54.793 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1221)
2019-01-26 18:18:54.793 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
2019-01-26 18:18:54.793 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
2019-01-26 18:18:54.793 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:   ... 16 more
2019-01-26 18:18:54.794 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err: Caused by: javax.net.ssl.SSLProtocolException: Read error: ssl=0x774f846f08: Failure in SSL library, usually a protocol error
2019-01-26 18:18:54.794 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err: error:10000412:SSL routines:OPENSSL_internal:SSLV3_ALERT_BAD_CERTIFICATE (external/boringssl/src/ssl/tls_record.cc:592 0x774f890d08:0x00000001)
2019-01-26 18:18:54.794 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.NativeCrypto.ENGINE_SSL_read_direct(Native Method)
2019-01-26 18:18:54.794 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.NativeSsl.readDirectByteBuffer(NativeSsl.java:521)
2019-01-26 18:18:54.794 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.ConscryptEngine.readPlaintextDataDirect(ConscryptEngine.java:1099)
2019-01-26 18:18:54.794 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.ConscryptEngine.readPlaintextDataHeap(ConscryptEngine.java:1119)
2019-01-26 18:18:54.795 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1091)
2019-01-26 18:18:54.795 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:     at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:841)
2019-01-26 18:18:54.795 31491-31771/xx.xxxxxxxxxx.xxxxxxxxx W/System.err:   ... 25 more

Error on server side:

 io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: null cert chain
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:646)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:581)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:460)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:748)Caused by: javax.net.ssl.SSLHandshakeException: null cert chain
    at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1441)
    at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535)
    at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:813)
    at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781)
    at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
    at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:294)
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1275)
    at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1177)
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1221)
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
    ... 16 moreCaused by: javax.net.ssl.SSLHandshakeException: null cert chain
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:201)
    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1672)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:309)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:297)
    at sun.security.ssl.ServerHandshaker.clientCertificate(ServerHandshaker.java:1942)
    at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:236)
    at sun.security.ssl.Handshaker.processLoop(Handshaker.java:984)
    at sun.security.ssl.Handshaker$1.run(Handshaker.java:924)
    at sun.security.ssl.Handshaker$1.run(Handshaker.java:921)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1379)
    at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1435)
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1343)
    ... 20 more

|

And of course: netty: 4.1.28.Final, Win10 and java version "1.8.0_151"

I tried netty 4.1.33.Final as well. Same result.

As you may have noticed I have ignored the test for hostname in the engine. This might be just for the server handshake and not for the client (private key) handshake. Please advice if you can on how to ignore the client hostname part of the handshake as well unless it is already done (I have tried doing it on both sides) as my application will not be able to support a dns name for the client. (It will be able to support it for the server but for the moment ignoring it).

I tried netty 4.1.34.Final-20190208.192045-20. Same result.
I tried netty 4.1.34.Final. Same result.
I tried netty 4.1.35.Final-SNAPSHOT. Same result.

Body:
I was facing this problem while performing two way SSL with Netty. I solved it by using following configuration:

public HttpClient getHttpClient(HttpClientProperties properties){

        // configure pool resources
        HttpClientProperties.Pool pool = properties.getPool();

        ConnectionProvider connectionProvider;
        if (pool.getType() == DISABLED) {
            connectionProvider = ConnectionProvider.newConnection();
        }
        else if (pool.getType() == FIXED) {
            connectionProvider = ConnectionProvider.fixed(pool.getName(),
                    pool.getMaxConnections(), pool.getAcquireTimeout());
        }
        else {
            connectionProvider = ConnectionProvider.elastic(pool.getName());
        }

        HttpClient httpClient = HttpClient.create(connectionProvider)
                .tcpConfiguration(tcpClient -> {

                    if (properties.getConnectTimeout() != null) {
                        tcpClient = tcpClient.option(
                                ChannelOption.CONNECT_TIMEOUT_MILLIS,
                                properties.getConnectTimeout());
                    }

                    // configure proxy if proxy host is set.
                    HttpClientProperties.Proxy proxy = properties.getProxy();

                    if (StringUtils.hasText(proxy.getHost())) {

                        tcpClient = tcpClient.proxy(proxySpec -> {
                            ProxyProvider.Builder builder = proxySpec
                                    .type(ProxyProvider.Proxy.HTTP)
                                    .host(proxy.getHost());

                            PropertyMapper map = PropertyMapper.get();

                            map.from(proxy::getPort).whenNonNull().to(builder::port);
                            map.from(proxy::getUsername).whenHasText()
                                    .to(builder::username);
                            map.from(proxy::getPassword).whenHasText()
                                    .to(password -> builder.password(s -> password));
                            map.from(proxy::getNonProxyHostsPattern).whenHasText()
                                    .to(builder::nonProxyHosts);
                        });
                    }
                    return tcpClient;
                });

        HttpClientProperties.Ssl ssl = properties.getSsl();
        if (ssl.getTrustedX509CertificatesForTrustManager().length > 0
                || ssl.isUseInsecureTrustManager()) {
            httpClient = httpClient.secure(sslContextSpec -> {
                // configure ssl
                SslContextBuilder sslContextBuilder = SslContextBuilder.forClient();

                X509Certificate[] trustedX509Certificates = ssl
                        .getTrustedX509CertificatesForTrustManager();
                if (trustedX509Certificates.length > 0) {
                    sslContextBuilder.trustManager(trustedX509Certificates);
                }
                else if (ssl.isUseInsecureTrustManager()) {
                    sslContextBuilder
                            .trustManager(InsecureTrustManagerFactory.INSTANCE);
                }

                sslContextSpec.sslContext(sslContextBuilder)
                        .defaultConfiguration(ssl.getDefaultConfigurationType())
                        .handshakeTimeout(ssl.getHandshakeTimeout())
                        .closeNotifyFlushTimeout(ssl.getCloseNotifyFlushTimeout())
                        .closeNotifyReadTimeout(ssl.getCloseNotifyReadTimeout())
                        .handlerConfigurator(
                                (handler)->{
                                    SSLEngine engine = handler.engine();
                                    //engine.setNeedClientAuth(true);
                                    SSLParameters params = new SSLParameters();
                                    List<SNIMatcher> matchers = new LinkedList<>();
                                    SNIMatcher matcher = new SNIMatcher(0) {

                                        @Override
                                        public boolean matches(SNIServerName serverName) {
                                            return true;
                                        }
                                    };
                                    matchers.add(matcher);
                                    params.setSNIMatchers(matchers);
                                    engine.setSSLParameters(params);
                                }
                        )
                ;
            });
        }

        return httpClient;

    }

The above code snippet uses handlerConfigurator provided by netty's Builder present inside the SslProvider class to customize the SslHandler to avoid hostname matching.


来源: stackexchange

问:
Does HTML Encoding a mailto tag stop spammers
I think its fairly common practice these days not to include a mailto: tag in your webpage.  I also realize that a better solution is to have a contact us form.

Recently I've seen some mailto tags that include and HTML encoded email address.  Is this an effective way to stop spammers from attacking an email address?

Example:

```<code><a href="mailto:newbi...">email me</a>
</code>```

Also, are there any other techniques that I've left out?  Which is the best method?

答:
Without making it hard for some of your users, you're only ever going to be able to stop lazy spammers — because you don't know ahead of time which you're looking at. So if you deliver something easily accessible to a legitimate user, you're delivering something easily accessible to a spammer.

My best advice is just to get a very good spam filter if you're going to put your address on the Web. No amount of href obfuscation will do as much good.

OwnerDisplayName: bendewey

ContentLicense: CC BY-SA 3.0

Body:
I think its fairly common practice these days not to include a mailto: tag in your webpage.  I also realize that a better solution is to have a contact us form.

Recently I've seen some mailto tags that include and HTML encoded email address.  Is this an effective way to stop spammers from attacking an email address?

Example:

email me

Also, are there any other techniques that I've left out?  Which is the best method?

LastEditorDisplayName: bendewey

OwnerDisplayName: Chuck

ContentLicense: CC BY-SA 2.5


问:
Can I do this with data.table without using the merge() function?
I work with very huge datasets and I'm trying to speed up my R code.
Here is an example of data :

```<code>dt <- data.table(id = c(100,101,102,103, 104), sex = c("m","f","m","m","f"),
value = c(32,14,32,03,03))

data look like this :
    id sex value
1: 100   m    32
2: 101   f    14
3: 102   m    32
4: 103   m     3
5: 104   f     3
</code>```

The final output I want :

```<code>   value f.value m.value f   m
1:     3    1       1    1   1
2:    14    1       NA   1   NA
3:    32    NA      2    NA  2
</code>```

The code I currently use :

```<code>dt_u <- unique(dt, by = c("id", "sex", "value"))
dt_u <- dt_u[, .(n = .N), keyby = .(value, sex)]
dt_u <- dcast(dt_u, value ~ sex, value.var = "n")
dt_t <- dt[, .(n = .N), keyby = .(value, sex)]
dt_t <- dcast(dt_t, value ~ sex, value.var = "n")
dt <- merge(dt_t, dt_u, by = "value", all = TRUE)
</code>```

The code is working great, the problem is that the merging of dt_u and dt_t can take a lot of time on 10GB + data. So my question is : Is it possible to get the same final output without having to "split" the data and then merging it?

Also I would like the answer to be in data.table if possible,
thank you.

EDIT : example and explaination. The ID represent a person, this person can go to the same location (value) more than one time. For this example you can say that each value represent a different city.

IE:

```<code>dt <- data.table(value = c(21,21,21,21,21,40,1,22,1,1,22, 22, 49,
49,21,21,1,1,1), id =
c(1000716624,1000722724,1000716624,1000746824,1001012024,
1002067324,1002743624,1002743645, 1002743636,
1002743423,1000716624,1000722724, 1000722724,1001012024,
1000716624,1000716624,1002743624,1002743624,1002743624), sex = c("f", "m",
"m", "m", "f", "f", "m", "f", "f", "m", "f", "m", "m", "f","f","f", "m",
"m", "m"))
</code>```

Output :

```<code> value   places_women   places_men  number_women   number_men
1:     1            1          5            1          2
2:    21            4          3            2          3
3:    22            2          1            2          1
4:    40            1         NA            1         NA
5:    49            1          1            1          1
</code>```

答:
This works for the second example (based on reverse engineering the desired output):

```<code>> dcast(dt, value ~ sex, value.var=list("value", "id"), fun=list(length, uniqueN), fill=NA)
   value value.1_length_f value.1_length_m id_uniqueN_f id_uniqueN_m
1:     1                1                5            1            2
2:    21                4                3            2            3
3:    22                2                1            2            1
4:    40                1               NA            1           NA
5:    49                1                1            1            1
</code>```

A more explicit description of what calculations are supposed to be in each column (and maybe more natural column names in the example) would help if this doesn't work for the full problem.

ContentLicense: CC BY-SA 4.0

Body:
I work with very huge datasets and I'm trying to speed up my R code.
Here is an example of data :

dt <- data.table(id = c(100,101,102,103, 104), sex = c("m","f","m","m","f"),
value = c(32,14,32,03,03))

data look like this :
    id sex value
1: 100   m    32
2: 101   f    14
3: 102   m    32
4: 103   m     3
5: 104   f     3

The final output I want :

   value f.value m.value f   m
1:     3    1       1    1   1
2:    14    1       NA   1   NA
3:    32    NA      2    NA  2

The code I currently use :

dt_u <- unique(dt, by = c("id", "sex", "value"))
dt_u <- dt_u[, .(n = .N), keyby = .(value, sex)]
dt_u <- dcast(dt_u, value ~ sex, value.var = "n")
dt_t <- dt[, .(n = .N), keyby = .(value, sex)]
dt_t <- dcast(dt_t, value ~ sex, value.var = "n")
dt <- merge(dt_t, dt_u, by = "value", all = TRUE)

The code is working great, the problem is that the merging of dt_u and dt_t can take a lot of time on 10GB + data. So my question is : Is it possible to get the same final output without having to "split" the data and then merging it?

Also I would like the answer to be in data.table if possible,
thank you.

EDIT : example and explaination. The ID represent a person, this person can go to the same location (value) more than one time. For this example you can say that each value represent a different city.

IE:

dt <- data.table(value = c(21,21,21,21,21,40,1,22,1,1,22, 22, 49,
49,21,21,1,1,1), id =
c(1000716624,1000722724,1000716624,1000746824,1001012024,
1002067324,1002743624,1002743645, 1002743636,
1002743423,1000716624,1000722724, 1000722724,1001012024,
1000716624,1000716624,1002743624,1002743624,1002743624), sex = c("f", "m",
"m", "m", "f", "f", "m", "f", "f", "m", "f", "m", "m", "f","f","f", "m",
"m", "m"))

Output :

 value   places_women   places_men  number_women   number_men
1:     1            1          5            1          2
2:    21            4          3            2          3
3:    22            2          1            2          1
4:    40            1         NA            1         NA
5:    49            1          1            1          1

Body:
This works for the second example (based on reverse engineering the desired output):

> dcast(dt, value ~ sex, value.var=list("value", "id"), fun=list(length, uniqueN), fill=NA)
   value value.1_length_f value.1_length_m id_uniqueN_f id_uniqueN_m
1:     1                1                5            1            2
2:    21                4                3            2            3
3:    22                2                1            2            1
4:    40                1               NA            1           NA
5:    49                1                1            1            1

A more explicit description of what calculations are supposed to be in each column (and maybe more natural column names in the example) would help if this doesn't work for the full problem.


问:
I would like to put a binary masking vector on the tensor 3D in the tensor flow
I would like to put a binary masking vector on the tensor 3D in the tensor flow.

For example, shape=(None, 3,4,1)

```<code>[[1,2,3,4][5,6,7,8][9,10,11,12]]
</code>```

I want to put a masking vector like

```<code>[[1,1,1,1] [1,1,1,1] [1,1,0,0]]
</code>```

=> my ideal result is

```<code>[[1,2,3,4][5,6,7,8][9,10,0,0]]
</code>```

I try it as follow.

```<code>output = tf.layers.conv2d(output, 1, [5, 5], strides=(9, 6), padding='valid')
output = tf.tanh(output)
aa = [1.0] * 10 + [0.0] *2
aa = aa * batch_size
bb = tf.constant(aa , shape =(batch_size , 3,4,1))
output = output * bb
</code>```

I want to put a mask vector behind the output.

output = tf.layers.conv2d(output, 1, [5, 5], strides=(9, 6), padding='valid')
output = tf.tanh(output)

## output shape is (batch size , 3,4,1)
## and then mask vector [[1,1,1,1] [1,1,1,1] [1,1,0,0]] 3x4
## (batch size , mask vector , 1)
## (batch size, 3, 4, 1)
# I want to put a mask vector behind the output.
# Include any research you've conducted
```

<p>I like this</p>

```masking = tf.sequence_mask([27]*19 + [17] , maxlen=27, dtype=tf.float32)
masking2 = tf.expand_dims(masking,axis = 0)
masking2 = tf.expand_dims(masking2,axis = -1)
mask = tf.tile(masking2, [batch_size, 1 , 1,1])
G = G * mask
```

答:
tf.sequence_mask should do what you need.

In your particular example it would be tf.sequence_mask([4, 4, 2], maxlen=4, dtype=tf.float32).

Body:
I would like to put a binary masking vector on the tensor 3D in the tensor flow.

For example, shape=(None, 3,4,1)

[[1,2,3,4][5,6,7,8][9,10,11,12]]

I want to put a masking vector like

[[1,1,1,1] [1,1,1,1] [1,1,0,0]]

=> my ideal result is

[[1,2,3,4][5,6,7,8][9,10,0,0]]

I try it as follow.

output = tf.layers.conv2d(output, 1, [5, 5], strides=(9, 6), padding='valid')
output = tf.tanh(output)
aa = [1.0] * 10 + [0.0] *2
aa = aa * batch_size
bb = tf.constant(aa , shape =(batch_size , 3,4,1))
output = output * bb

I want to put a mask vector behind the output.

output = tf.layers.conv2d(output, 1, [5, 5], strides=(9, 6), padding='valid')
output = tf.tanh(output)

## output shape is (batch size , 3,4,1)
## and then mask vector [[1,1,1,1] [1,1,1,1] [1,1,0,0]] 3x4
## (batch size , mask vector , 1)
## (batch size, 3, 4, 1)
# I want to put a mask vector behind the output.
# Include any research you've conducted

I like this

masking = tf.sequence_mask([27]*19 + [17] , maxlen=27, dtype=tf.float32)
masking2 = tf.expand_dims(masking,axis = 0)
masking2 = tf.expand_dims(masking2,axis = -1)
mask = tf.tile(masking2, [batch_size, 1 , 1,1])
G = G * mask


问:
Is it possible to make repo searchable, but not visible to the public?
Github newbie here.

If my understanding is correct, there are only 2 visibility modes for a repository: "Public" and "Private". However, neither of these modes satisfy my needs. What I need is something in between.

I would like to make a repo searchable, meaning when a user visits my profile, they see that a repo exists, and also private, meaning they don't actually see the code when visiting my profile (but they can request permission to view it).

Such a feature/functionality would be really nifty as I don't want just anybody to have access to my code, but would like to be able to share it with potential employers.

Is it possible to do such a thing? Any help is greatly appreciated!

Many thanks in advance!
KC

答: Not really you can take the andvantages of Premium with private repos.</p>


问:
ng new want to write to '/Users/USERNAME/.npm/_cacache/index-v5/aa/9a'
I got this weird error, that if I want to create a new Angular app with the CLI, it is going to fail because of the write permission.
Here is the npm error log inside of the console.

```<code>npm ERR! path /Users/USERNAME/.npm/_cacache/index-v5/aa/9a
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! syscall mkdir
npm ERR! Error: EACCES: permission denied, mkdir '/Users/USERNAME/.npm/_cacache/index-v5/aa/9a'
npm ERR!  [OperationalError: EACCES: permission denied, mkdir '/Users/USERNAME/.npm/_cacache/index-v5/aa/9a'] {
npm ERR!   cause: [Error: EACCES: permission denied, mkdir '/Users/USERNAME/.npm/_cacache/index-v5/aa/9a'] {
npm ERR!     errno: -13,
npm ERR!     code: 'EACCES',
npm ERR!     syscall: 'mkdir',
npm ERR!     path: '/Users/USERNAME/.npm/_cacache/index-v5/aa/9a'
npm ERR!   },
npm ERR!   isOperational: true,
npm ERR!   stack: 'Error: EACCES: permission denied, mkdir ' +
npm ERR!     "'/Users/USERNAME/.npm/_cacache/index-v5/aa/9a'",
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'mkdir',
npm ERR!   path: '/Users/USERNAME/.npm/_cacache/index-v5/aa/9a',
npm ERR!   parent: 'AngularPortfolio'
npm ERR! }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator (though this is not recommended).

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/USERNAME/.npm/_logs/2019-05-29T20_08_02_308Z-debug.log
Package install failed, see above.
</code>```

I couldn't figure out why and what is happening...

I'm using angular cli 8v but I got the same problem with minor versions. So I guess angular is not involved in that.

If I run the command with sudo, no probleme, but that doesn't seem right! Even with sudo I got a lot of error message like

```<code>gyp ERR! configure error
gyp ERR! stack Error: EACCES: permission denied, mkdir '/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/build'
gyp ERR! System Darwin 18.6.0
gyp ERR! command "/usr/local/bin/node" "/Users/USERNAME/.npm-global/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--module=/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/lib/binding/Release/node-v72-darwin-x64/fse.node" "--module_name=fse" "--module_path=/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/lib/binding/Release/node-v72-darwin-x64" "--napi_version=4" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v72"
gyp ERR! cwd /Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents
gyp ERR! node -v v12.3.1
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute '/usr/local/bin/node /Users/USERNAME/.npm-global/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/lib/binding/Release/node-v72-darwin-x64/fse.node --module_name=fse --module_path=/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/lib/binding/Release/node-v72-darwin-x64 --napi_version=4 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v72' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:200:13)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:1021:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:283:5)
node-pre-gyp ERR! System Darwin 18.6.0
node-pre-gyp ERR! command "/usr/local/bin/node" "/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/node_modules/node-pre-gyp/bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents
node-pre-gyp ERR! node -v v12.3.1
node-pre-gyp ERR! node-pre-gyp -v v0.12.0
node-pre-gyp ERR! not ok
</code>```

```<code>Failed to execute '/usr/local/bin/node /Users/USERNAME/.npm-global/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/lib/binding/Release/node-v72-darwin-x64/fse.node --module_name=fse --module_path=/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/lib/binding/Release/node-v72-darwin-x64 --napi_version=4 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v72' (1)

> core-js@2.6.9 postinstall /Users/USERNAME/Documents/Workspace/lol/node_modules/babel-runtime/node_modules/core-js
> node scripts/postinstall || echo "ignore"

> core-js@2.6.9 postinstall /Users/USERNAME/Documents/Workspace/lol/node_modules/karma/node_modules/core-js
> node scripts/postinstall || echo "ignore"

> @angular/cli@8.0.0 postinstall /Users/USERNAME/Documents/Workspace/lol/node_modules/@angular/cli
> node ./bin/postinstall/script.js

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 (node_modules/fsevents):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 install: `node install`
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1

added 1010 packages from 1041 contributors and audited 19008 packages in 41.655s
found 0 vulnerabilities
</code>```

I also got the problem with installing packages. npm install only works with sudo.
Node -v is 12
npm -v is 6.9.0

答:
The solution layed in my .npmrc. I don't know why, but there where the global prefix set to .npm...
Now there is no permission issues anymore!</p>

Body:
I got this weird error, that if I want to create a new Angular app with the CLI, it is going to fail because of the write permission.
Here is the npm error log inside of the console.

npm ERR! path /Users/USERNAME/.npm/_cacache/index-v5/aa/9a
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! syscall mkdir
npm ERR! Error: EACCES: permission denied, mkdir '/Users/USERNAME/.npm/_cacache/index-v5/aa/9a'
npm ERR!  [OperationalError: EACCES: permission denied, mkdir '/Users/USERNAME/.npm/_cacache/index-v5/aa/9a'] {
npm ERR!   cause: [Error: EACCES: permission denied, mkdir '/Users/USERNAME/.npm/_cacache/index-v5/aa/9a'] {
npm ERR!     errno: -13,
npm ERR!     code: 'EACCES',
npm ERR!     syscall: 'mkdir',
npm ERR!     path: '/Users/USERNAME/.npm/_cacache/index-v5/aa/9a'
npm ERR!   },
npm ERR!   isOperational: true,
npm ERR!   stack: 'Error: EACCES: permission denied, mkdir ' +
npm ERR!     "'/Users/USERNAME/.npm/_cacache/index-v5/aa/9a'",
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'mkdir',
npm ERR!   path: '/Users/USERNAME/.npm/_cacache/index-v5/aa/9a',
npm ERR!   parent: 'AngularPortfolio'
npm ERR! }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator (though this is not recommended).

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/USERNAME/.npm/_logs/2019-05-29T20_08_02_308Z-debug.log
Package install failed, see above.

I couldn't figure out why and what is happening...

I'm using angular cli 8v but I got the same problem with minor versions. So I guess angular is not involved in that.

If I run the command with sudo, no probleme, but that doesn't seem right! Even with sudo I got a lot of error message like

gyp ERR! configure error
gyp ERR! stack Error: EACCES: permission denied, mkdir '/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/build'
gyp ERR! System Darwin 18.6.0
gyp ERR! command "/usr/local/bin/node" "/Users/USERNAME/.npm-global/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--module=/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/lib/binding/Release/node-v72-darwin-x64/fse.node" "--module_name=fse" "--module_path=/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/lib/binding/Release/node-v72-darwin-x64" "--napi_version=4" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v72"
gyp ERR! cwd /Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents
gyp ERR! node -v v12.3.1
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute '/usr/local/bin/node /Users/USERNAME/.npm-global/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/lib/binding/Release/node-v72-darwin-x64/fse.node --module_name=fse --module_path=/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/lib/binding/Release/node-v72-darwin-x64 --napi_version=4 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v72' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:200:13)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:1021:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:283:5)
node-pre-gyp ERR! System Darwin 18.6.0
node-pre-gyp ERR! command "/usr/local/bin/node" "/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/node_modules/node-pre-gyp/bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents
node-pre-gyp ERR! node -v v12.3.1
node-pre-gyp ERR! node-pre-gyp -v v0.12.0
node-pre-gyp ERR! not ok

Failed to execute '/usr/local/bin/node /Users/USERNAME/.npm-global/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/lib/binding/Release/node-v72-darwin-x64/fse.node --module_name=fse --module_path=/Users/USERNAME/Documents/Workspace/lol/node_modules/fsevents/lib/binding/Release/node-v72-darwin-x64 --napi_version=4 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v72' (1)

> core-js@2.6.9 postinstall /Users/USERNAME/Documents/Workspace/lol/node_modules/babel-runtime/node_modules/core-js
> node scripts/postinstall || echo "ignore"

> core-js@2.6.9 postinstall /Users/USERNAME/Documents/Workspace/lol/node_modules/karma/node_modules/core-js
> node scripts/postinstall || echo "ignore"

> @angular/cli@8.0.0 postinstall /Users/USERNAME/Documents/Workspace/lol/node_modules/@angular/cli
> node ./bin/postinstall/script.js

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 (node_modules/fsevents):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 install: `node install`
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1

added 1010 packages from 1041 contributors and audited 19008 packages in 41.655s
found 0 vulnerabilities

I also got the problem with installing packages. npm install only works with sudo.
Node -v is 12
npm -v is 6.9.0


问:
How to calculate month with fraction in Access VBA
I need to write a function that receives two arguments startdate and enddate where it returns the difference between these two dates.

Say that function is monthfractiondiff(startdate,enddate)

e.g. monthfractiondiff(15/01/2016,15/02/2016) will return
((31-15)/31)+(15/29) = 1.0333704116

monthfractiondiff(15/11/2018,15/02/2019) will return
((30-15)/30)+1+1+(15/28) = 3.0357142857

I vainly tried https://access-programmers.co.uk/forums/showthread.php?t=113589

答:
As the day count of months varies, you have to count by days to get as close as possible, as there never (except within a month or within July/August or December/January) will be an exact value:

```<code>' Rounds by default to two decimals, as more decimals has no meaning
' due to the varying count of days of a month.
' Optionally, don't round, by setting Round2 to False.
'
Public Function TotalMonths( _
    ByVal Date1 As Date, _
    ByVal Date2 As Date, _
    Optional Round2 As Boolean = True) _
    As Double

    Dim Months      As Double
    Dim Part1       As Double
    Dim Part2       As Double
    Dim Fraction    As Double
    Dim Result      As Double

    Months = DateDiff("m", Date1, Date2)
    Part1 = (Day(Date1) - 1) / DaysInMonth(Date1)
    Part2 = (Day(Date2) - 1) / DaysInMonth(Date2)

    If Round2 = True Then
        ' Round to two decimals.
        Fraction = (-Part1 + Part2) * 100
        Result = Months + Int(Fraction + 0.5) / 100
    Else
        Result = Months - Part1 + Part2
    End If

    TotalMonths = Result

End Function

' Returns the count of days of the month of Date1.
'
' 2016-02-14. Gustav Brock, Cactus Data ApS, CPH.
'
Public Function DaysInMonth( _
    ByVal Date1 As Date) _
    As Integer

    Const MaxDateValue  As Date = #12/31/9999#
    Const MaxDayValue   As Integer = 31

    Dim Days    As Integer

    If DateDiff("m", Date1, MaxDateValue) = 0 Then
        Days = MaxDayValue
    Else
        Days = Day(DateSerial(Year(Date1), Month(Date1) + 1, 0))
    End If

    DaysInMonth = Days

End Function
</code>```

Results:

```<code>? TotalMonths(#2016/01/15#, #2016/02/15#)
 1.03

? TotalMonths(#2018/11/15#, #2019/02/15#)
 3.03
</code>```

Body:
As the day count of months varies, you have to count by days to get as close as possible, as there never (except within a month or within July/August or December/January) will be an exact value:

' Rounds by default to two decimals, as more decimals has no meaning
' due to the varying count of days of a month.
' Optionally, don't round, by setting Round2 to False.
'
Public Function TotalMonths( _
    ByVal Date1 As Date, _
    ByVal Date2 As Date, _
    Optional Round2 As Boolean = True) _
    As Double

    Dim Months      As Double
    Dim Part1       As Double
    Dim Part2       As Double
    Dim Fraction    As Double
    Dim Result      As Double

    Months = DateDiff("m", Date1, Date2)
    Part1 = (Day(Date1) - 1) / DaysInMonth(Date1)
    Part2 = (Day(Date2) - 1) / DaysInMonth(Date2)

    If Round2 = True Then
        ' Round to two decimals.
        Fraction = (-Part1 + Part2) * 100
        Result = Months + Int(Fraction + 0.5) / 100
    Else
        Result = Months - Part1 + Part2
    End If

    TotalMonths = Result

End Function

' Returns the count of days of the month of Date1.
'
' 2016-02-14. Gustav Brock, Cactus Data ApS, CPH.
'
Public Function DaysInMonth( _
    ByVal Date1 As Date) _
    As Integer

    Const MaxDateValue  As Date = #12/31/9999#
    Const MaxDayValue   As Integer = 31

    Dim Days    As Integer

    If DateDiff("m", Date1, MaxDateValue) = 0 Then
        Days = MaxDayValue
    Else
        Days = Day(DateSerial(Year(Date1), Month(Date1) + 1, 0))
    End If

    DaysInMonth = Days

End Function

Results:

? TotalMonths(#2016/01/15#, #2016/02/15#)
 1.03

? TotalMonths(#2018/11/15#, #2019/02/15#)
 3.03


问:
Why does my bash script flag this awk substring command as a syntactic error when it works in the terminal?
I'm trying to extract a list of dates from a series of links using lynx's dump function and piping the output through grep and awk. This operation works successfully in the terminal and outputs dates accurately. However, when it is placed into a shell script, bash claims a syntax error:

```<code>Scripts/ETC/PreD.sh: line 18: syntax error near unexpected token `('
Scripts/ETC/PreD.sh: line 18: ` lynx --dump "$link" | grep -m 1 Date | awk '{print substr($0,10)}' >> dates.txt'
</code>```

For context, this is part of a while-read loop in which $link is being read from a file. Operations undertaken inside this while-loop when the awk command is removed are all successful, as are similar while-loops that include other awk commands.

I know that either I'm misunderstanding how bash handles variable substitution, or how bash handles awk commands, or some combination of the two. Any help would be immensely appreciated.

EDIT: Shellcheck is divided on this, the website version finds no error, but my downloaded version provides error SC1083, which says:

```<code>This { is literal. Check expression (missing ;/\n?) or quote it.
</code>```

A check on the Shellcheck GitHub page provides this:

```<code>This error is harmless when the curly brackets are supposed to be literal, in e.g. awk {'print $1'}.
However, it's cleaner and less error prone to simply include them inside the quotes: awk '{print $1}'.
</code>```

Script follows:

```<code>#!/bin/bash

while read -u 4 link
do
        IFS=/ read a b c d e <<< "$link"
        echo "$e" >> 1.txt
        lynx --dump "$link" | grep -A 1 -e With: | tr -d [:cntrl:][:digit:][] | sed 's/\With//g' | awk '{print substr($0,10)}' | sed 's/\(.*\),/\1'\ and'/' | tr -s ' ' >> 2.txt
        lynx --dump "$link" | grep -m 1 Date | awk '{print substr($0,10)}' >> dates.txt
done 4< links.txt
</code>```

答:
In sed command you have unmatched ', due to unquoted '.

In awk script your have constant zero length variable.

From gawk manual:

substr(string, start [, length ])

Return a length-character-long substring of string, starting at character number start. The first character of a string is character
  number one.48 For example, substr("washington", 5, 3) returns "ing".

If length is not present, substr() returns the whole suffix of string that begins at character number start. For example,
  substr("washington", 5) returns "ington". The whole suffix is also
  returned if length is greater than the number of characters remaining
  in the string, counting from character start.

If start is less than one, substr() treats it as if it was one. (POSIX doesn’t specify what to do in this case: BWK awk acts this way,
  and therefore gawk does too.) If start is greater than the number of
  characters in the string, substr() returns the null string. Similarly,
  if length is present but less than or equal to zero, the null string
  is returned.

Also I suggest you combine grep|awk|sed|tr into single awk script. And debug the awk script with printouts.

From:

```<code>lynx --dump "$link" | grep -A 1 -e With: | tr -d [:cntrl:][:digit:][] | sed 's/\With//g' | awk '{print substr($0,10,length)}' | sed 's/\(.*\),/\1'\ and'/' | tr -s ' ' >> 2.txt
</code>```

To:

```<code>lynx --dump "$link" | awk '/With/{found=1;next}found{found=0;print sub(/\(.*\),/,"& and",gsub(/ +/," ",substr($0,10)))}' >> 2.txt
</code>```

From:

```<code>lynx --dump "$link" | grep -m 1 Date | awk '{print substr($0,10,length)}' >> dates.txt
</code>```

To:

```<code>lynx --dump "$link" | awk '/Date/{print substr($0,10)}' >> dates.txt
</code>```

Body:
I'm trying to extract a list of dates from a series of links using lynx's dump function and piping the output through grep and awk. This operation works successfully in the terminal and outputs dates accurately. However, when it is placed into a shell script, bash claims a syntax error:

Scripts/ETC/PreD.sh: line 18: syntax error near unexpected token `('
Scripts/ETC/PreD.sh: line 18: ` lynx --dump "$link" | grep -m 1 Date | awk '{print substr($0,10)}' >> dates.txt'

For context, this is part of a while-read loop in which $link is being read from a file. Operations undertaken inside this while-loop when the awk command is removed are all successful, as are similar while-loops that include other awk commands.

I know that either I'm misunderstanding how bash handles variable substitution, or how bash handles awk commands, or some combination of the two. Any help would be immensely appreciated.

EDIT: Shellcheck is divided on this, the website version finds no error, but my downloaded version provides error SC1083, which says:

This { is literal. Check expression (missing ;/\n?) or quote it.

A check on the Shellcheck GitHub page provides this:

This error is harmless when the curly brackets are supposed to be literal, in e.g. awk {'print $1'}.
However, it's cleaner and less error prone to simply include them inside the quotes: awk '{print $1}'.

Script follows:

#!/bin/bash

while read -u 4 link
do
        IFS=/ read a b c d e <<< "$link"
        echo "$e" >> 1.txt
        lynx --dump "$link" | grep -A 1 -e With: | tr -d [:cntrl:][:digit:][] | sed 's/\With//g' | awk '{print substr($0,10)}' | sed 's/\(.*\),/\1'\ and'/' | tr -s ' ' >> 2.txt
        lynx --dump "$link" | grep -m 1 Date | awk '{print substr($0,10)}' >> dates.txt
done 4< links.txt

Body:
In sed command you have unmatched ', due to unquoted '.

In awk script your have constant zero length variable.

From gawk manual:

substr(string, start [, length ])

Return a length-character-long substring of string, starting at character number start. The first character of a string is character
  number one.48 For example, substr("washington", 5, 3) returns "ing".

If length is not present, substr() returns the whole suffix of string that begins at character number start. For example,
  substr("washington", 5) returns "ington". The whole suffix is also
  returned if length is greater than the number of characters remaining
  in the string, counting from character start.

If start is less than one, substr() treats it as if it was one. (POSIX doesn’t specify what to do in this case: BWK awk acts this way,
  and therefore gawk does too.) If start is greater than the number of
  characters in the string, substr() returns the null string. Similarly,
  if length is present but less than or equal to zero, the null string
  is returned.

Also I suggest you combine grep|awk|sed|tr into single awk script. And debug the awk script with printouts.

From:

lynx --dump "$link" | grep -A 1 -e With: | tr -d [:cntrl:][:digit:][] | sed 's/\With//g' | awk '{print substr($0,10,length)}' | sed 's/\(.*\),/\1'\ and'/' | tr -s ' ' >> 2.txt

To:

lynx --dump "$link" | awk '/With/{found=1;next}found{found=0;print sub(/\(.*\),/,"& and",gsub(/ +/," ",substr($0,10)))}' >> 2.txt

From:

lynx --dump "$link" | grep -m 1 Date | awk '{print substr($0,10,length)}' >> dates.txt

To:

lynx --dump "$link" | awk '/Date/{print substr($0,10)}' >> dates.txt


问:
AJAX commit change to database without submit
This cat has been skinned several ways in SO, but I can't seem to stretch any in to a way that works for my example.

I have created a table consisting of three columns. The third column is a text box that will display a placeholder of 'Not Installed', or a date. I need an onChange event to trigger a PHP post script to send the entered value to the database.

Independently, the following works:

HTML

```<code><div class="row">
    <form action="./functions/post_Updates.php" method="Post">
         <input name="RAID">
         <input name="InstalledOn">
         <button type="submit" value="go">go </button>
    </form>
</div>
</code>```

PHP

```<code>$RAID = $_POST['RAID'];
$InstalledOn = $_POST['InstalledOn'];

echo "<p>$RAID</p>";
echo "<p>$InstalledOn</p>";

$tsql = "insert into dbo.test (test) values ('$InstalledOn $RAID')";
</code>```

Upon submit, the TEST table gets updated.

I set up my table elements in the following manner (as part of a PHP function to generate the table itself:
HTML

```<code><input type=\"text\" class=\"InstalledOn_Input\" name=\"$r_RAAID\" onChange=\"DynamicUpdateCP($r_RAAID,this.value);\" placeholder=\"$r_CPInstalledOnDate\"></input>
</code>```

JS

```<code>function DynamicUpdateCP(id,value){
    $.ajax({
        data: {RAID: id, InstalledOn: value},
        url: './functions/post_Updates.php',
        type: 'POST'
    });
    alert(id + ' has changed to ' + value);

}
</code>```

If I remove the $.ajax portion, the alert() fires as expected. Currently, though, editing a field and, either pressing return or changing focus, does nothing at all.

Any guidance on the missing link, here, would be phenomenal. Thank you.

@Pointy
I reformatted the portion of the onchange=... to be:

```<code>onChange=\"DynamicUpdateCP(\"$r_RAAID\",\"this.value\");\"'
</code>```

the error I get, in my debug console, is:

Uncaught SyntaxError: Unexpected end of input

jQuery import statement:

```<code><script rel="javascript" type="text/javascript"  src="http://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
</code>```

答:
EDIT: was calling the JQuery script using HTTP, not HTTPS. Updated to below:

```<code><script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
</code>```

Changed the version of jquery to 3.4.1 and the ajax call worked perfectly.

Body:
This cat has been skinned several ways in SO, but I can't seem to stretch any in to a way that works for my example.

I have created a table consisting of three columns. The third column is a text box that will display a placeholder of 'Not Installed', or a date. I need an onChange event to trigger a PHP post script to send the entered value to the database.

Independently, the following works:

HTML

    <form action="./functions/post_Updates.php" method="Post">
         <input name="RAID">
         <input name="InstalledOn">
         <button type="submit" value="go">go </button>
    </form>

PHP

$RAID = $_POST['RAID'];
$InstalledOn = $_POST['InstalledOn'];

echo "
$RAID
";
echo "
$InstalledOn
";

$tsql = "insert into dbo.test (test) values ('$InstalledOn $RAID')";

Upon submit, the TEST table gets updated.

I set up my table elements in the following manner (as part of a PHP function to generate the table itself:
HTML

<input type=\"text\" class=\"InstalledOn_Input\" name=\"$r_RAAID\" onChange=\"DynamicUpdateCP($r_RAAID,this.value);\" placeholder=\"$r_CPInstalledOnDate\"></input>

JS

function DynamicUpdateCP(id,value){
    $.ajax({
        data: {RAID: id, InstalledOn: value},
        url: './functions/post_Updates.php',
        type: 'POST'
    });
    alert(id + ' has changed to ' + value);

}

If I remove the $.ajax portion, the alert() fires as expected. Currently, though, editing a field and, either pressing return or changing focus, does nothing at all.

Any guidance on the missing link, here, would be phenomenal. Thank you.

@Pointy
I reformatted the portion of the onchange=... to be:

onChange=\"DynamicUpdateCP(\"$r_RAAID\",\"this.value\");\"'

the error I get, in my debug console, is:

Uncaught SyntaxError: Unexpected end of input

jQuery import statement:

Body:
EDIT: was calling the JQuery script using HTTP, not HTTPS. Updated to below:

<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>

Changed the version of jquery to 3.4.1 and the ajax call worked perfectly.


问:
AWS CodeBuild as non-root user
Is there a way to drop root user on AWS CodeBuild?
We are building a Yocto project that fails on CodeBuild if we're root (Bitbake sanity check).

Our desperate approach doesn't work either:

```<code>...

build:
  commands:
    - chmod -R 777 $(pwd)/ && chown -R builder $(pwd)/ && su -c "$(pwd)/make.sh" -s /bin/bash builder
...
</code>```

Fails with:

```<code>bash: /codebuild/output/src624711770/src/.../make.sh: Permission denied
</code>```

Any idea how we could run this a non-root?

答: Sigh, so I came across this question and I am disappointed that there is no good or simple answer to this problem.  There are many, many processes that strongly discourage running as root like composer and others that will flat-out refuse like  wp-cli.  If you are using the Ubuntu "standard image" provided by AWS, then there appears to be an existing user in the /etc/passwd file, dockremap:x:1000:1000::/home/dockremap:/bin/sh.  I think this user is for userns-remap in docker and I am not sure about it's availability.  The other option that astonishingly hasn't been mentioned is running useradd -N -G users develop to create a new user in the container.  It is far simpler than spinning up a custom container for something so trivial.

Title: AWS CodeBuild as non-root user

Body:
Is there a way to drop root user on AWS CodeBuild?
We are building a Yocto project that fails on CodeBuild if we're root (Bitbake sanity check).

Our desperate approach doesn't work either:

...

build:
  commands:
    - chmod -R 777 $(pwd)/ && chown -R builder $(pwd)/ && su -c "$(pwd)/make.sh" -s /bin/bash builder
...

Fails with:

bash: /codebuild/output/src624711770/src/.../make.sh: Permission denied

Any idea how we could run this a non-root?


问:
optimize sql query on 5B records table in postgresql database
My postgresql database has the table electrical_measurement which contains approximately 5 billion records. I have indexes on every column. I am trying to perform the following query but it never ends. Wondering how I can modify it to run faster.

```<code>SELECT
    em.id AS em_id,
    em.test_board_id_in,
    em.die,
    tvt_net.name,
    mb_pad_map.x,
    mb_pad_map.y,
    em.temperature,
    em.timestamp,
    em.avg_meas_voltage
FROM electrical_measurement AS em
INNER JOIN main_board_pad_map AS mb_pad_map
    ON em.net_id_in = mb_pad_map.net_id
INNER JOIN tvt_net
    ON em.net_id_in = tvt_net.id
WHERE em.assembly_id = 1
AND em.net_id_in IN
 (SELECT em.net_id_in
  FROM electrical_measurement AS em
  WHERE em.assembly_id = 1
  AND em.avg_meas_voltage > 0
  GROUP BY em.net_id_in)
ORDER BY em.timestamp
</code>```

This is the result from EXPLAIN:

```<code>-------------------------------------------------------------------------------------------------------------------------------------------------------
 Gather Merge  (cost=373158311.30..573643901.29 rows=1718327938 width=63)
   Workers Planned: 2
   ->  Sort  (cost=373157311.28..375305221.20 rows=859163969 width=63)
         Sort Key: em."timestamp"
         ->  Hash Join  (cost=84935808.04..171830022.94 rows=859163969 width=63)
               Hash Cond: (em.net_id_in = mb_pad_map.net_id)
               ->  Hash Join  (cost=84935424.26..161155613.60 rows=118993479 width=41)
                     Hash Cond: (em.net_id_in = em_1.net_id_in)
                     ->  Parallel Bitmap Heap Scan on electrical_measurement em  (cost=2996320.29..78903135.78 rows=118993479 width=37)
                           Recheck Cond: (assembly_id = 1)
                           ->  Bitmap Index Scan on electrical_measurement_assembly_id_idx  (cost=0.00..2924924.21 rows=285584350 width=0)
                                 Index Cond: (assembly_id = 1)
                     ->  Hash  (cost=81939087.68..81939087.68 rows=1303 width=4)
                           ->  HashAggregate  (cost=81939061.62..81939074.65 rows=1303 width=4)
                                 Group Key: em_1.net_id_in
                                 ->  Bitmap Heap Scan on electrical_measurement em_1  (cost=2953194.68..81656356.93 rows=113081878 width=4)
                                       Recheck Cond: (assembly_id = 1)
                                       Filter: (avg_meas_voltage > '0'::numeric)
                                       ->  Bitmap Index Scan on electrical_measurement_assembly_id_idx  (cost=0.00..2924924.21 rows=285584350 width=0)
                                             Index Cond: (assembly_id = 1)
               ->  Hash  (cost=266.17..266.17 rows=9408 width=38)
                     ->  Hash Join  (cost=42.32..266.17 rows=9408 width=38)
                           Hash Cond: (mb_pad_map.net_id = tvt_net.id)
                           ->  Seq Scan on main_board_pad_map mb_pad_map  (cost=0.00..199.08 rows=9408 width=16)
                           ->  Hash  (cost=26.03..26.03 rows=1303 width=22)
                                 ->  Seq Scan on tvt_net  (cost=0.00..26.03 rows=1303 width=22)
(26 rows)
</code>```

Do you have any suggestions?
Thanks

答: The subquery is the killer. Need to remove it and do it another way. Thanks for your comments. </p>

Body:
My postgresql database has the table electrical_measurement which contains approximately 5 billion records. I have indexes on every column. I am trying to perform the following query but it never ends. Wondering how I can modify it to run faster.

SELECT
    em.id AS em_id,
    em.test_board_id_in,
    em.die,
    tvt_net.name,
    mb_pad_map.x,
    mb_pad_map.y,
    em.temperature,
    em.timestamp,
    em.avg_meas_voltage
FROM electrical_measurement AS em
INNER JOIN main_board_pad_map AS mb_pad_map
    ON em.net_id_in = mb_pad_map.net_id
INNER JOIN tvt_net
    ON em.net_id_in = tvt_net.id
WHERE em.assembly_id = 1
AND em.net_id_in IN
 (SELECT em.net_id_in
  FROM electrical_measurement AS em
  WHERE em.assembly_id = 1
  AND em.avg_meas_voltage > 0
  GROUP BY em.net_id_in)
ORDER BY em.timestamp

This is the result from EXPLAIN:

-------------------------------------------------------------------------------------------------------------------------------------------------------
 Gather Merge  (cost=373158311.30..573643901.29 rows=1718327938 width=63)
   Workers Planned: 2
   ->  Sort  (cost=373157311.28..375305221.20 rows=859163969 width=63)
         Sort Key: em."timestamp"
         ->  Hash Join  (cost=84935808.04..171830022.94 rows=859163969 width=63)
               Hash Cond: (em.net_id_in = mb_pad_map.net_id)
               ->  Hash Join  (cost=84935424.26..161155613.60 rows=118993479 width=41)
                     Hash Cond: (em.net_id_in = em_1.net_id_in)
                     ->  Parallel Bitmap Heap Scan on electrical_measurement em  (cost=2996320.29..78903135.78 rows=118993479 width=37)
                           Recheck Cond: (assembly_id = 1)
                           ->  Bitmap Index Scan on electrical_measurement_assembly_id_idx  (cost=0.00..2924924.21 rows=285584350 width=0)
                                 Index Cond: (assembly_id = 1)
                     ->  Hash  (cost=81939087.68..81939087.68 rows=1303 width=4)
                           ->  HashAggregate  (cost=81939061.62..81939074.65 rows=1303 width=4)
                                 Group Key: em_1.net_id_in
                                 ->  Bitmap Heap Scan on electrical_measurement em_1  (cost=2953194.68..81656356.93 rows=113081878 width=4)
                                       Recheck Cond: (assembly_id = 1)
                                       Filter: (avg_meas_voltage > '0'::numeric)
                                       ->  Bitmap Index Scan on electrical_measurement_assembly_id_idx  (cost=0.00..2924924.21 rows=285584350 width=0)
                                             Index Cond: (assembly_id = 1)
               ->  Hash  (cost=266.17..266.17 rows=9408 width=38)
                     ->  Hash Join  (cost=42.32..266.17 rows=9408 width=38)
                           Hash Cond: (mb_pad_map.net_id = tvt_net.id)
                           ->  Seq Scan on main_board_pad_map mb_pad_map  (cost=0.00..199.08 rows=9408 width=16)
                           ->  Hash  (cost=26.03..26.03 rows=1303 width=22)
                                 ->  Seq Scan on tvt_net  (cost=0.00..26.03 rows=1303 width=22)
(26 rows)

Do you have any suggestions?
Thanks


问:
Missing assembly references
I have cleaned the solution. Even deleted contents of the bin folder. But I still get

Type or namespace name 'Mvc' does not exists in the namespace 'System.Web'

The type or namespace name 'Kendo' could not be found (are you missing a using directive or an assembly reference?)

How do I resolve this?

答: Try Cata Hotea's solution.  I can't tell if you have the reference from your screenshot.  It stops. Once System.Web.Mvc is installed.  Clean your solution. Close your project.  Go to the file system.  Delete everything within the packages directory.  Then delete your hidden .vs.  Then open the solution again within VS. You should get a pop up about needing to download the packages or you can just build the project.  Also, be sure to add your Kendo.Mvc reference to the Web.config within your Views directory in XML tag system.web -> pages -> namespaces -> add. Also, note adding or changing System.Web.Mvc or Razor does not always update the Web.config version numbers. Be sure those are correct to avoid warnings.</p>

Title: Missing assembly references


问:
Include empty responses in form
I am trying using HTML, Jquery, and Node.js to develop an online test for my students. Currently, when I submit the form, the server gets a json object (using body-parser) that includes only the fields that were answered. However, I also need to be able to see which ones were not answer. For example, if my student answers questions 1, 2, 3 but not 4, the server shows the following:

{Q1: a, Q2:b, Q3:c}

I want to get:

{Q1: a, Q2:b, Q3:c, Q4:''}

I tried looking online, however most posts explain how to get rid of the empty responses and not how to include them.

This is what I have right now

      <form id="Form1" action="/Form1" method="POST">

                 1.
                A </labe> <input type="radio" name="Q1" value="A">
                B </labe> <input type="radio" name="Q1" value="B">
                 C </labe> <input type="radio" name="Q1" value="C">
                 D </labe> <input type="radio" name="Q1" value="D">

                 2.
                 A </labe> <input type="radio" name="Q2" value="A">
                 B </labe> <input type="radio" name="Q2" value="B">
                 C </labe> <input type="radio" name="Q2" value="C">
                 D </labe> <input type="radio" name="Q2" value="D">

                 3.
                 A </labe> <input type="radio" name="Q3" value="A">
                 B </labe> <input type="radio" name="Q3" value="B">
                 C </labe> <input type="radio" name="Q3" value="C">
                 D </labe> <input type="radio" name="Q3" value="D">

                 4.
                 A </labe> <input type="radio" name="Q4" value="A">
                 B </labe> <input type="radio" name="Q4" value="B">
                 C </labe> <input type="radio" name="Q4" value="C">
                 D </labe> <input type="radio" name="Q4" value="D">

            <input type="submit" value="Submit">

      </form>```
</div>
</div>
</p>

<p>and in the server I have,</p>

```app.post('/Form1', urlencodedParser, function(request,response){ console.log(request.body); })
```

答:
You could add a hidden radio button for each question that is marked as checked:

```<code><input type="radio" name="Q4" value="" style="display: none;" checked>
</code>```

Title: Include empty responses in form

Body:
I am trying using HTML, Jquery, and Node.js to develop an online test for my students. Currently, when I submit the form, the server gets a json object (using body-parser) that includes only the fields that were answered. However, I also need to be able to see which ones were not answer. For example, if my student answers questions 1, 2, 3 but not 4, the server shows the following:

{Q1: a, Q2:b, Q3:c}

I want to get:

{Q1: a, Q2:b, Q3:c, Q4:''}

I tried looking online, however most posts explain how to get rid of the empty responses and not how to include them.

This is what I have right now

      <form id="Form1" action="/Form1" method="POST">

                 1.
                A </labe> <input type="radio" name="Q1" value="A">
                B </labe> <input type="radio" name="Q1" value="B">
                 C </labe> <input type="radio" name="Q1" value="C">
                 D </labe> <input type="radio" name="Q1" value="D">

                 2.
                 A </labe> <input type="radio" name="Q2" value="A">
                 B </labe> <input type="radio" name="Q2" value="B">
                 C </labe> <input type="radio" name="Q2" value="C">
                 D </labe> <input type="radio" name="Q2" value="D">

                 3.
                 A </labe> <input type="radio" name="Q3" value="A">
                 B </labe> <input type="radio" name="Q3" value="B">
                 C </labe> <input type="radio" name="Q3" value="C">
                 D </labe> <input type="radio" name="Q3" value="D">

                 4.
                 A </labe> <input type="radio" name="Q4" value="A">
                 B </labe> <input type="radio" name="Q4" value="B">
                 C </labe> <input type="radio" name="Q4" value="C">
                 D </labe> <input type="radio" name="Q4" value="D">

            <input type="submit" value="Submit">

      </form>

and in the server I have,

app.post('/Form1', urlencodedParser, function(request,response){ console.log(request.body); })

Body:
You could add a hidden radio button for each question that is marked as checked:

<input type="radio" name="Q4" value="" style="display: none;" checked>


问:
What is the maximum amount of data i passed through querystring of jquery ajax?
What is the maximum amount of data i passed through querystring of jquery ajax?</p>

答:
The maximum url length for any http request varies from browser to browser.  Jquery won't limit you, but the browser the request is being made from will.

See this post:
What is the maximum length of a URL in different browsers?