Cache Proxy Server Returning 404 with www.google.com
Question: I have a homework assignment which involves implementing a proxy cache server
in Python for web pages. Here is my implementation of it

    
    
    from socket import *
    import sys
    
    def main():
        #Create a server socket, bind it to a port and start listening
        tcpSerSock = socket(AF_INET, SOCK_STREAM) #Initializing socket
        tcpSerSock.bind(("", 8030)) #Binding socket to port
        tcpSerSock.listen(5) #Listening for page requests
        while True:
            #Start receiving data from the client
            print 'Ready to serve...'
            tcpCliSock, addr = tcpSerSock.accept()
            print 'Received a connection from:', addr
            message = tcpCliSock.recv(1024)
            print message
    
            #Extract the filename from the given message
            filename = ""
            try:
                filename = message.split()[1].partition("/")[2].replace("/", "")
            except:
                continue
            fileExist = False
    
            try: #Check whether the file exists in the cache
                f = open(filename, "r")
                outputdata = f.readlines()
                fileExist = True
                #ProxyServer finds a cache hit and generates a response message
                tcpCliSock.send("HTTP/1.0 200 OK\r\n")
                tcpCliSock.send("Content-Type:text/html\r\n")
                for data in outputdata:
                    tcpCliSock.send(data)
                print 'Read from cache'
            except IOError: #Error handling for file not found in cache
                if fileExist == False:
    
                    c = socket(AF_INET, SOCK_STREAM) #Create a socket on the proxyserver
    
                    try:
                        srv = getaddrinfo(filename, 80)
                        c.connect((filename, 80)) #https://docs.python.org/2/library/socket.html
                        # Create a temporary file on this socket and ask port 80 for
                        # the file requested by the client
                        fileobj = c.makefile('r', 0)
                        fileobj.write("GET " + "http://" + filename + " HTTP/1.0\r\n")
                        # Read the response into buffer
                        buffr = fileobj.readlines()
                        # Create a new file in the cache for the requested file.
                        # Also send the response in the buffer to client socket and the
                        # corresponding file in the cache
                        tmpFile = open(filename,"wb")
                        for data in buffr:
                            tmpFile.write(data)
                            tcpCliSock.send(data)
                    except:
                        print "Illegal request"
                else: #File not found
                    print "404: File Not Found"
            tcpCliSock.close() #Close the client and the server sockets
    
    main()
    

I configured my browsers to use my proxy server like so

[![enter image description
here](http://i.stack.imgur.com/o2FjV.jpg)](http://i.stack.imgur.com/o2FjV.jpg)

But my problem when I run it is that no matter what web page I try to access
it returns a 404 error with the initial connection and then a connection reset
error with subsequent connections. I have no idea why so any help would be
greatly appreciated, thanks!


Answer: There are quite a number of issues with your code.

Your URL parser is quite cumbersome. Instead of the line

    
    
    filename = message.split()[1].partition("/")[2].replace("/", "")
    

I would use

    
    
    import re
    parsed_url = re.match(r'GET\s+http://(([^/]+)(.*))\sHTTP/1.*$', message)
    local_path = parsed_url.group(3)
    host_name = parsed_url.group(2)
    filename = parsed_url.group(1)
    

If you catch an exception there, you should probably throw an error because it
is a request your proxy doesn't understand (e.g. a POST).

When you assemble your request to the destination server, you then use

    
    
    fileobj.write("GET {object} HTTP/1.0\n".format(object=local_path))
    fileobj.write("Host: {host}\n\n".format(host=host_name))
    

You should also include some of the header lines from the original request
because they can make a major difference to the returned content.

Furthermore, you currently cache the entire response with all header lines, so
you should not add your own when serving from cache.

What you have doesn't work, anyway, because there is no guarantee that you
will get a 200 and `text/html` content. You should check the response code and
only cache if you did indeed get a 200.

