Data from a Python script to URL as JSON
Question: I've spent a lot of time on this but still can't seem to get it to work. The
task is - I have to send system stats to a URL and the script is supposed to
pull it, convert the namedtuple of each cpu stats of a machine and then send
them all in 1 single POST request as JSON. The connection must close once the
data has been sent.

For the '1 single POST request' functionality, I added the latter function
(senddata_to_server) in the script. Without it (with the connection details
simply listed there without a function) , when I ran it on Mac/Windows/Linux,
it used to return all the namedtuples 1 by 1 and then a '200 OK' and then go
on printing 'Connection refused' forever. Now when I run it, it just hangs
there without returning anything.

(I have asked this question earlier ( [HTTP Post request with Python
JSON](http://stackoverflow.com/questions/18754405/http-post-request-with-
python-json) ) but I need to have the 'params' inside the loop and the
connection details outside it.

    
    
    import psutil 
    import socket
    import time
    import sample
    import json
    import httplib
    import urllib
    
    serverHost = sample.host
    port = sample.port
    
    thisClient = socket.gethostname()
    currentTime = int(time.time())
    s = socket.socket()
    s.connect((serverHost, port))
    cpuStats = psutil.cpu_times_percent(percpu=True)
    
    
    
    def loop_thru_cpus():
        while True:
            global cpuStats
    
            cpuStats = "/n".join([json.dumps(stats._asdict()) for stats in cpuStats])
    
            try:
    
                command = 'put cpu.usr ' + str(currentTime) + " " + str(cpuStats[0]) + "host ="+thisClient+ "/n"
                s.sendall(command)
                command = 'put cpu.nice ' + str(currentTime) + " " + str(cpuStats[1]) + "host ="+ thisClient+ "/n"
                s.sendall(command)
                command = 'put cpu.sys ' + str(currentTime) + " " + str(cpuStats[2]) + "host ="+ thisClient+ "/n"
                s.sendall(command)
                command = 'put cpu.idle ' + str(currentTime) + " " + str(cpuStats[3]) + "host ="+ thisClient+ "/n"
                s.sendall(command)
                params = urllib.urlencode({'cpuStats': cpuStats, 'deviceKey': 1234, 'timeStamp': str(currentTime)})
                return params
                print cpuStats
    
            except IndexError:
                continue
            except socket.error:
                print "Connection refused"
                continue
    
            finally:
                s.close()
    
    
    def senddata_to_server():
        x = loop_thru_cpus()
        headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
        conn = httplib.HTTPConnection(serverHost, port)
        conn.request = ("POST", "", x.params, headers)
        response = conn.response()
        print response.status, response. reason
        print x.cpuStats
        conn.close()
    loop_thru_cpus()
    senddata_to_server()
    


Answer: > > Given the task and the code/logic here, what am I doing wrong?

I can't quite tell what your task is, but here are some things you may be
doing wrong:

  * You are connecting to the server directly (via `socket.connect()`) and through the framework (via `HTTPlib.connect()`)
  * You misspelled newline: it should be `'\n'`, not `'/n'`
  * You have a while loop that can only execute once (because you `return` in the middle of it).
  * You have a print statement after your return statement
  * You are sending malformed `put` commands to the web server
  * You call `loop_thru_cpus()` twice
  * You set the content-type incorrectly to `application/json` \-- you aren't sending well-formed json.
  * You aren't sending a url to `HTTPlib.HTTPConnection.request()` (may be allowed in practice, disallowed in the documentation)
  * You aren't invoking `conn.request()` correctly -- get rid of `=`
  * In [the documentation](http://docs.python.org/2/library/httplib.html) it says to call `conn.getresponse()`, not `conn.response()`

Here is a program that hopefully does what you ask for:

    
    
    import psutil 
    import socket
    import time
    import json
    import httplib
    import urllib
    
    # httpbin provides an echo service at http://httpbin.org/post
    serverHost = 'httpbin.org'
    port = 80
    url = 'http://httpbin.org/post'
    
    # My psutil only has cpu_times, not cpu_times_percent
    cpuStats = psutil.cpu_times(percpu=True)
    
    # Convert each namedTuple to a json string
    cpuStats = [json.dumps(stats._asdict()) for stats in cpuStats]
    
    # Convert each json string to the form required by the assignment
    cpuStats = [urllib.urlencode({'cpuStats':stats, 'deviceKey':1234}) for stats in cpuStats]
    
    # Join stats together, one per line
    cpuStats = '\n'.join(cpuStats)
    
    # Send the data ...
    
    # connect
    conn = httplib.HTTPConnection(serverHost, port)
    
    # Send the data
    conn.request("POST", url, cpuStats)
    
    # Check the response, should be 200
    response = conn.getresponse()
    print response.status, response.reason
    
    # httpbin.org provides an echo service -- what did we send?
    print response.read()
    
    conn.close()
    

