Can't log in to website with Python requests session module
Question: I am just starting out with web scraping. For my first project, I'm trying to
log into artofproblemsolving.com using requests.Session() and access another
user's account. Here is my code:

    
    
    import requests
    
    LOGIN_URL = 'https://www.artofproblemsolving.com/Forum/ucp.php?mode=login'
    DATA_URL = 'https://www.artofproblemsolving.com/Forum/memberlist.php?mode=viewprofile&u=90586'
    
    payload = {
        'username': '{{my_username}}',
        'password': '{{my_password}}'
    }
    
    with requests.Session() as s:
        s.post(LOGIN_URL, data=payload)
        r = s.get(DATA_URL)
        print r.text
    

But when I run this in terminal, the output HTML is from the login page, not
the user's profile that I'm trying to grab. I double-checked that LOGIN_URL is
the POST action in the login form, that 'username' and 'password' are the
names of the form items, and that my login information is correct.

All help is appreciated, thank you!


Answer: I'm not sure this is direct reason for the problem. But there are another
parameter that the form send to login action.

Beside `username`, `password`, also pass following parameters:

    
    
    'username': username,
    'password': password,
    'login': 'Login'.
    'sid': sid,   # You need to parse the login page to get sid
    'redirect': 'index.php',
    

