POST to .jsp website with Python requests library
Question: I'm using Python 3.3 and Requests 2.2.1.

I'm trying to POST to a website ending in .jsp, which then changes to .doh
ending. Using the same basic requests code outline I'm able to successfully
login and scrape other websites, but the javascript part on this site is not
working. This is my code:

    
    
    import requests
    
    url = 'https://prodpci.etimspayments.com/pbw/include/sanfrancisco/input.jsp'
    payload = {'plateNumber':'notshown', 'statePlate':'CA'} #tried CA and California
    
    s = requests.Session() #Tried 'session' and 'Session' following different advice
    post = s.post(url, data=payload)
    r = s.get('https://prodpci.etimspayments.com/pbw/include/sanfrancisco/input.jsp')
    print(r.text)
    

Finally, when manually entering data into the webpage through firefox browser,
the page changes and url becomes
<https://prodpci.etimspayments.com/pbw/inputAction.doh>, which only has contet
if you are redirected there after typing in license plate.

From the printed text, I know I'm getting content from the page as it would be
without POSTing anything, but I need the content for the page once I've POSTed
the payload. For the POST payload, do I need to include something like
'submit':'submit' to simulate clicking the search button?

Am I doing the GET request from the right url, considering the url I POST to?


Answer: You're making POST request and after that another GET request and this is why
you get the same page with the form.

    
    
    response = s.post(url, data=payload)
    print(response.text)
    

Also if you check the form markup, you'll find its action is
`/pbw/inputAction.doh` and additionally the form sends a few parameters from
`hidden` inputs. Therefore you should use that URL in your request and
probably the values from `hidden` inputs.

![enter image description here](http://i.stack.imgur.com/9ZZbA.png)

With the next code I'm able to retrieve the same response as via regular
request in browser:

    
    
    import requests
    
    url = 'https://prodpci.etimspayments.com/pbw/inputAction.doh'
    payload = {
        'plateNumber': 'notshown',
        'statePlate': 'CA',
        'requestType': 'submit',
        'clientcode': 19,
        'requestCount': 1,
        'clientAccount': 5,
    }
    
    s = requests.Session()
    response = s.post(url, data=payload)
    print(response.text)
    

The same you can see in browser after same request via the form:

    
    
    ...
    <td colspan="2"> <li class="error">Plate is not found</li></td>
    ...
    

