How to handle ssl connections in raw Python socket?
Question: I'm writing a program to download a given webpage. I need to only use raw
python sockets for all the connection due to some restriction. So I make a
socket connection to a given domain (the Host field in the response header of
an object) and then send the GET request on this. Now when the url is a https
url, I think I need to first do the SSL handshake (because otherwise I'm
getting non-200 OK responses from the server and other error responses
mentioning P3P policies). I inspected curl's response to check how it's able
to successfully download while I'm not, turns out curl first does the SSL
handshake (that's all the difference). curl is always able to successfully
download a given object, the only difference always being the SSL handshake it
does.

So I'm wondering how to do the SSL handshake in raw python sockets? Basically
I want as easy a solution which allows me to do the minimum besides using raw
sockets.


Answer: Here is an example of a TCP client with SLL.

Not sure if it's the best way to download a web page but it should answer your
question "SSL handshake in raw python socket".

You will probably have to adapt the struct.pack/unpack but you get the general
idea:

    
    
    import socket
    import ssl
    import struct
    import binascii
    import sys
    
    class NotConnectedException(Exception):
        def __init__(self, message=None, node=None):
            self.message = message
            self.node = node
    
    class DisconnectedException(Exception):
        def __init__(self, message=None, node=None):
            self.message = message
            self.node = node
    
    class Connector:
        def __init__(self):
            pass
    
        def is_connected(self):
            return (self.sock and self.ssl_sock)
    
        def open(self, hostname, port, cacert):
            self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            self.ssl_sock = ssl.wrap_socket(self.sock, ca_certs=cacert, cert_reqs=ssl.CERT_REQUIRED)
    
            if hostname == socket.gethostname():
                ipaddress = socket.gethostbyname_ex(hostname)[2][0]
                self.ssl_sock.connect((ipaddress, port))
            else:
                self.ssl_sock.connect((hostname, port))
    
            self.sock.setsockopt(socket.SOL_TCP, socket.TCP_NODELAY, 1)
    
        def close(self):
            if self.sock: self.sock.close()
            self.sock = None
            self.ssl_sock = None
    
        def send(self, buffer):
            if not self.ssl_sock: raise NotConnectedException("Not connected (SSL Socket is null)")
            self.ssl_sock.sendall(struct.pack('L', len(buffer)))
            self.ssl_sock.sendall(buffer)
    
        def receive(self):
            if not self.ssl_sock: raise NotConnectedException("Not connected (SSL Socket is null)")
            data_size_buffer = self.ssl_sock.recv(4)
    
            if len(data_size_buffer) <= 0:
                raise DisconnectedException()
    
            data_size = struct.unpack('L', data_size_buffer)[0]
            received_size = 0
            data_buffer = ""
    
            while received_size < data_size:
                chunk = self.ssl_sock.recv(1024)
                data_buffer += chunk
                received_size += len(chunk)
    
            return data_buffer
    

Then you use the class like this:

    
    
        connector = Connector.Connector()
        connector.open(server_ip, server_port, path_to_the_CA_cert.pem)
        connector.send(your_data)
        response = connector.receive()
        connector.close()
    

