python web crawler with ticket authentication
Question: The website I'm trying to crawl have an login page like:

    
    
    <form method="post" action="/login" enctype="multipart/form-data">
    <table><tbody><tr><td>Name</td> 
                      <td><input type="text" name="user"></td>                  
                  </tr> 
                  <tr><td>Password</td> 
                      <td><input type="password" name="password"></td>
                  </tr>
           </tbody>
     </table>
     <input type="hidden" name="request_uri" value="/index.html">
     <input type="submit" name="Log In" value="Log In">
    <p></p></form>
    

An access ticket will be generated by the server after the login data (account
and password) have been successfully validated, containing information that
will grant access to restricted areas of the site.

This ticket, along with additional data, will be wrapped into a cookie and
sent to the browser. This cookie does not contain any life-time or expiration
information, and will thus be discarded when the session ends.

How can I login like an user by python?


Answer: Use the `requests` module and a POST to login in, then save the cookie. I
would recommend using `Fiddler2` to capture the traffic to the website and
then build your response such that the payload in your Python code matches the
payload expected by the server. Your code would look something like this:

    
    
    import requests
    
    url = 'http://mywebsite.com'
    payload = {'user': 'my_usersname', 'password': 'my_password'}
    r = requests.post(url, params=payload)
    print r.text  # for Python 3.x use print(r.text)
    print r.cookies
    

