title: C++ OpenSSL Fails to perform handshake when accepting in non-blocking mode. What is the proper way?
body: <p>I'm trying to implement OpenSSL into my application which uses raw C sockets and the only issue I'm having is the SSL_accept / SSL_connect part of the code which starts the KeyExchange phase but does not seem to complete it on the serverside.</p>
  
  <p>I've had a look at countless websites and Q&amp;A's here on StackOverflow to get myself through the OpenSSL API since this is basically the first time I'm attempting to implement SSL into an application but the only thing I could not find yet was how to properly manage failed handshakes.</p>
  
  <p>Basically, running process A which serves as a server will listen for incoming connections. Once I run process B, which acts as a client, it will successfully connect to process A but SSL_accept (on the server) fails with error code -2 SSL_ERROR_WANT_READ.</p>
  
  <p>According to <a href="https://stackoverflow.com/questions/6263060/openssl-handshake-failed">openssl handshake failed</a>, the problem is "easily" worked around by calling SSL_accept within a loop until it finally returns 1 (It successfully connects and completes the handshake). However, I do not believe that this is the proper way of doing things as it looks like a dirty trick. The reason for why I believe it is a dirty trick is because I tried to run a small application I found on <a href="https://www.cs.utah.edu/~swalton/listings/articles/" rel="nofollow noreferrer">https://www.cs.utah.edu/~swalton/listings/articles/</a> (ssl_client and ssl_server) and magically, everything works just fine. There are no multiple calls to SSL_accept and the handshake is completed right away.</p>
  
  <p>Here's some code where I'm accepting the SSL connection on the server:</p>
  
  <pre><code>if (SSL_accept(conn.ssl) == -1)
              {
                  fprintf(stderr, "Connection failed.\n");
                  fprintf(stderr, "SSL State: %s [%d]\n", SSL_state_string_long(conn.ssl), SSL_state(conn.ssl));
                  ERR_print_errors_fp(stderr);
                  PrintSSLError(conn.ssl, -1, "SSL_accept");
                  return -1;
              }
              else
              {
                  fprintf(stderr, "Connection accepted.\n");
                  fprintf(stderr, "Server -&gt; Client handshake completed");
              }
  </code></pre>
  
  <p>This is the output of PrintSSLError:</p>
  
  <pre><code>SSL State: SSLv3 read client hello B [8465]
  [DEBUG] SSL_accept : Failed with return -1
  [DEBUG]     SSL_get_error() returned : 2
  [DEBUG]     Error string : error:00000002:lib(0):func(0):system lib
  [DEBUG]     ERR_get_error() returned : 0
  [DEBUG]     errno returned : Resource temporarily unavailable
  </code></pre>
  
  <p>And here's the client side snippet which connects to the server:</p>
  
  <pre><code>if (SSL_connect(conn.ssl) == -1)
              {
                  fprintf(stderr, "Connection failed.\n");
                  ERR_print_errors_fp(stderr);
                  PrintSSLError(conn.ssl, -1, "SSL_connect");
                  return -1;
              }
              else
              {
                  fprintf(stderr, "Connection established.\n");
                  fprintf(stderr, "Client -&gt; Server handshake completed");
                  PrintSSLInfo(conn.ssl);
              }
  </code></pre>
  
  <p>The connection is successfully enstablished client-side (SSL_connect does not return -1) and PrintSSLInfo outputs:</p>
  
  <pre><code>Connection established.
  Cipher: DHE-RSA-AES256-GCM-SHA384
  SSL State: SSL negotiation finished successfully [3]
  </code></pre>
  
  <p>And this is how I wrap the C Socket into SSL:</p>
  
  <pre><code>SSLConnection conn;
          conn.fd = fd;
          conn.ctx = sslContext;
  
          conn.ssl = SSL_new(conn.ctx);
          SSL_set_fd(conn.ssl, conn.fd);
  </code></pre>
  
  <p>The code snippet here resides within a function that takes a file-descriptor of the accepted incoming connection on the raw socket and the SSL Context to use.</p>
  
  <p>To initialize the SSL Contexts I use TLSv1_2_server_method() and TLSv1_2_client_method(). Yes, I know that this will prevent clients from connecting if they do not support TLS 1.2 but this is exactly what I want. Whoever connects to my application will have to do it through my client anyway.</p>
  
  <p>Either way, what am I doing wrong? I'd like to avoid loops in the authentication phase to avoid possible hang ups/slow downs of the application due to unexpected infinite loops since OpenSSL does not specify how many attempts it might take.</p>
  
  <p>The workaround that worked, but that I'd like to avoid, is this:</p>
  
  <pre><code>while ((accept = SSL_accept(conn.ssl)) != 1)
  </code></pre>
  
  <p>And inside the while loop I check for the return code stored inside accept.</p>
  
  <p>Things I've tried to workaround the SSL_ERROR_WANT_READ error:</p>
  
  <ol>
  <li>Added usleep(50) inside the while loop (still takes several cycles to complete)</li>
  <li>Added SSL_do_handshake(conn.ssl) after SSL_connect and SSL_accept (didn't change anything on the end-result)</li>
  <li>Had a look at the code shown on roxlu.com (search on Google for "Using OpenSSL with memory BIOs - Roxlu") to guide me through the handshaking phase but since I'm new to this, and I don't directly use BIOs in my code but simply wrap my native C sockets into SSL, it was kind of confusing. I'm also unable to re-write the Networking part of the application as it'd would be too much work for me right now.</li>
  </ol>
  
  <p>I've done some tests with the openssl command-line as well to troubleshoot the issue but it gives no error. The handshake appears to be successful as no errors such as:</p>
  
  <pre><code>24069864:error:1409E0E5:SSL routines:ssl3_write_bytes:ssl handshake failure:s3_pkt.c:656
  </code></pre>
  
  <p>appear. Here's the whole output of the command</p>
  
  <pre><code>openssl s_client -connect IP:Port -tls1_2 -prexit -msg
  </code></pre>
  
  <p><a href="http://pastebin.com/9u1bfuf4" rel="nofollow noreferrer">http://pastebin.com/9u1bfuf4</a></p>
  
  <p>Things to note:
  1. I'm using the latest OpenSSL version 1.0.2h
  2. Application runs on a Unix system
  3. Using self-signed certificates to encrypt the network traffic</p>
  
  <p>Thanks everyone who's going to help me out.</p>
  
  <p>Edit:
  I forgot to mention that the sockets are in non-blocking mode since the application serves multiple clients in one-go. Though, client-side they are in blocking mode.</p>
  
  <p>Edit2: 
  Leaving this here for future reference: jmarshall.com/stuff/handling-nbio-errors-in-openssl.html</p>