Twisted Python, TLS and client/server certificate authentication error
Question: I've been learning Twisted as best I can, but together with limited TLS
knowledge it's proving challenging. I'm trying to write (ultimately) an SMTP
server that can send and receive messages both as plain text, or via TLS
depending on requirements of a specific message to be sent / received.

My sample server code (thus far, just handling the TLS connection, no SMTP
bits yet!) is borrowed from
<http://twistedmatrix.com/documents/11.0.0/core/howto/ssl.html#auto5> and
looks like:

    
    
    from OpenSSL import SSL
    from twisted.internet import reactor, ssl
    from twisted.internet.protocol import ServerFactory
    from twisted.protocols.basic import LineReceiver
    
    class TLSServer(LineReceiver):
        def lineReceived(self, line):
            print "received: " + line
    
            if line == "STARTTLS":
                print "-- Switching to TLS"
                self.sendLine('READY')
                ctx = ServerTLSContext(
                    privateKeyFileName='SSCerts/serverkey.pem',
                    certificateFileName='SSCerts/servercert.pem',
                    )
                self.transport.startTLS(ctx, self.factory)
    
    
    class ServerTLSContext(ssl.DefaultOpenSSLContextFactory):
        def __init__(self, *args, **kw):
            kw['sslmethod'] = SSL.TLSv1_METHOD
            ssl.DefaultOpenSSLContextFactory.__init__(self, *args, **kw)
    
    if __name__ == '__main__':
        factory = ServerFactory()
        factory.protocol = TLSServer
        reactor.listenTCP(8000, factory)
        reactor.run()
    

while the client is borrowed from
<http://twistedmatrix.com/documents/14.0.0/core/howto/ssl.html#starttls-
client> and looks like:

    
    
    from twisted.internet import ssl, endpoints, task, protocol, defer
    from twisted.protocols.basic import LineReceiver
    from twisted.python.modules import getModule
    
    class StartTLSClient(LineReceiver):
        def connectionMade(self):
            self.sendLine("plain text")
            self.sendLine("STARTTLS")
    
        def lineReceived(self, line):
            print("received: " + line)
            if line == "READY":
                self.transport.startTLS(self.factory.options)
                self.sendLine("secure text")
                self.transport.loseConnection()
    
    @defer.inlineCallbacks
    def main(reactor):
        factory = protocol.Factory.forProtocol(StartTLSClient)
        certData = getModule(__name__).filePath.sibling('servercert.pem').getContent()
        factory.options = ssl.optionsForClientTLS(
            u"example.com", ssl.PrivateCertificate.loadPEM(certData)
        )
        endpoint = endpoints.HostnameEndpoint(reactor, 'localhost', 8000)
        startTLSClient = yield endpoint.connect(factory)
    
        done = defer.Deferred()
        startTLSClient.connectionLost = lambda reason: done.callback(None)
        yield done
    
    if __name__ == "__main__":
        import starttls_client
        task.react(starttls_client.main)
    

But when I have the server listening, and I run the client I get:

    
    
        /usr/lib64/python2.6/site-packages/twisted/internet/endpoints.py:30: DeprecationWarning: twisted.internet.interfaces.IStreamClientEndpointStringParser was deprecated in Twisted 14.0.0: This interface has been superseded by IStreamClientEndpointStringParserWithReactor.
      from twisted.internet.interfaces import (
    main function encountered error
    Traceback (most recent call last):
      File "starttls_client.py", line 33, in <module>
        task.react(starttls_client.main)
      File "/usr/lib64/python2.6/site-packages/twisted/internet/task.py", line 875, in react
        finished = main(_reactor, *argv)
      File "/usr/lib64/pytho
    
    n2.6/site-packages/twisted/internet/defer.py", line 1237, in unwindGenerator
            return _inlineCallbacks(None, gen, Deferred())
        --- <exception caught here> ---
          File "/usr/lib64/python2.6/site-packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
            result = g.send(result)
          File "/root/Robot/Twisted/starttls_client.py", line 22, in main
            u"example.com", ssl.PrivateCertificate.loadPEM(certData)
          File "/usr/lib64/python2.6/site-packages/twisted/internet/_sslverify.py", line 619, in loadPEM
            return Class.load(data, KeyPair.load(data, crypto.FILETYPE_PEM),
          File "/usr/lib64/python2.6/site-packages/twisted/internet/_sslverify.py", line 725, in load
            return Class(crypto.load_privatekey(format, data))
          File "build/bdist.linux-x86_64/egg/OpenSSL/crypto.py", line 2010, in load_privatekey
    
      File "build/bdist.linux-x86_64/egg/OpenSSL/_util.py", line 22, in exception_from_error_queue
    
    OpenSSL.crypto.Error: []
    

The strange thing is - I know the certificate and key are fine - I have other
"dummy" code (not pasted here, I figured this post is long enough!!) that uses
them for validation just fine. Can anyone explain where the code above falls
over? I'm at a loss...

Thanks :)


Answer: So it looks like there is a bug in the sample code found at:
<http://twistedmatrix.com/documents/14.0.0/core/howto/ssl.html>

Looking at the example "echoclient_ssl.py" there is the line:

    
    
    authority = ssl.Certificate.loadPEM(certData)
    

However, the equivalent bit of code in the "starttls_client.py" example code
is:

    
    
    ssl.PrivateCertificate.loadPEM(certData)
    

PrivateCertificate on the client side? Even with my limited understanding of
TLS, this seems wrong. Indeed, I modified my code to remove the "Private"...
and the error above disappears!

As I say, my knowledge and understanding is growing here - but this certainly
seems to be the issue / solution to my question!

