Using requests function in python to submit data to a website and call back a response
Question: I am trying to use the requests function in python to post the text content of
a text file to a website, submit the text for analysis on said website, and
pull the results back in to python. I have read through a number of responses
here and on other websites, but have not yet figured out how to correctly
modify the code to a new website.

I'm familiar with beautiful soup so pulling in webpage content and removing
HTML isn't an issue, its the submitting the data that I don't understand.

My code currently is:

    
    
    import requests
    
    fileName = "texttoAnalyze.txt"        
    fileHandle = open(fileName, 'rU');
    url_text = fileHandle.read()
    
    url = "http://www.webpagefx.com/tools/read-able/"
    payload = {'value':url_text}
    r = requests.post(url, payload)
    print r.text
    

This code comes back with the html of the website, but hasn't recognized the
fact that I'm trying to a submit a form.

Any help is appreciated. Thanks so much.


Answer: You need to send the same request the website is sending, usually you can get
these with web debugging tools (like chrome/firefox developer tools).

In this case the url the request is being sent to is:
`http://www.webpagefx.com/tools/read-able/check.php`

With the following params:
`tab=Test+by+Direct+Link&directInput=SOME_RANDOM_TEXT`

So your code should look like this:

    
    
    url = "http://www.webpagefx.com/tools/read-able/check.php"
    payload = {'directInput':url_text, 'tab': 'Test by Direct Link'}
    r = requests.post(url, data=payload)
    print r.text
    

Good luck!

