Submitting offline forms using python
Question: I am trying to extract information from my college website using python. Here
is the link of the website.

<http://studzone.psgtech.edu/CommonPage.aspx>

I have the exam results page locally saved. I want to know how to submit data
using the local file and get the resulting page using python. I've looked into
urllib2, requests, mechanize frameworks. But haven't got any useful
information on submitting data from a local HTML file. Thanks in advance.

Back note: The reason I saved it locally is, the website uses a specific token
for each detail such as Time-Table, Results, etc. So we need to send requests
from that specific page such as Time-Table, Results. i.e. A student can access
only one detail at a time and there is no separate page for that.

Edit:

I've use following code to obtain the web page and store it in a object. Is
there anyway to post the form in that webpage stored in the page variable? I
can edit the field such that value="something".

    
    
    import urllib2
    import requests
    
    page = urllib2.urlopen("http://google.com/").read()
    c = requests.post(page, data) //is something like this possible?
    


Answer: I have 3 files. `login.php`, `auth.php`, `index.php`. Direct access to
`index.php` is disabled. If user is not authenticated, request for `index.php`
is redirected to `login.php`.

    
    
    >>> url = 'http://127.0.0.1/adminpanel/admin/index.php'
    >>> import requests
    >>> r = requests.get(url)
    >>> print r.text
    
    <!DOCTYPE html>
    <html>
    <head>
        <title> Login </title>
    </head>
    <body>
            <form method="POST" action="auth.php">
                <input type="text" name="username">
                <input type="password" name="password">
                <input type="submit" name="submit" value="submit">                  
            </form>
    
    
    </body>
    </html>
    >>> print r.url
    http://127.0.0.1/adminpanel/admin/login.php
    

There is no authentication logic in login.php it is just a form. The
authentication process is held in auth.php. Let's send a post request with
some data to this url.

    
    
    >>> auth_url = "http://127.0.0.1/adminpanel/admin/auth.php"
    >>> params = {'username':'superadmin', 'password':'geometry123', 'submit':'submit'}
    >>> r = requests.post(auth_url, data=params)
    >>> print r.text[:200]
    
    
    
    <!DOCTYPE html>
    <html>
    <head>
    <meta charset="utf-8">
        <title>
            Admin Panel
        </title>
    <link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css" rel="style
    >>> print r.url
    http://127.0.0.1/adminpanel/admin/index.php
    

So, you have to: determine names of form inputs to send a post request. If
authentication will be successfull, you'll be redirected to protected page.
And with other libraries (`beautifulsoup`, `urrlib3`) , you can extract the
data you want.

