online web form access using python
Question: I am trying to access websites that require login information. For practice I
tried getting into Hotmail just to see if it would work. I have no idea if
this code is right. With trial and error I have the code "running", but it
isn't working still.. Can someone help me out?

Thanks, Brandon

    
    
        Python 3.4.0 (v3.4.0:04f714765c13, Mar 16 2014, 19:25:23) [MSC v.1600 64 bit (AMD64)] on win32
    Type "copyright", "credits" or "license()" for more information.
    
    
    
    import urllib.request
    import webbrowser
    
    url = 'http://hotmail.com'
    data = urllib.parse.urlencode({'idDiv_PWD_UsernameExample' : 'email','idDiv_PWD_PasswordExample' : 'password'})
    binary_data = data.encode('utf8')
    results = urllib.request.urlopen(url, binary_data)
    html = results.geturl()
    print (html)
    


Answer: # General Considerations

**Cookies** will possibly be used during the whole login process. Then the
following steps can become necessary:

  1. Load the login page to get the initial cookie.
  2. Load the login page again, sending login data and the cookie from step 1.
  3. Follow redirections after successful login.

But thanks to "batteries included", it's not that hard, I just tried the
following example with the Horde Groupware:

    
    
    import urllib.request
    import http.cookiejar
    import urllib.parse
    
    cookiejar = http.cookiejar.CookieJar()
    
    loginpage = "https://example.com/login.php"
    
    opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cookiejar))
    
    data = {'user': 'username', 'pw': 'password'}   # + what else might be required
    
    binData = urllib.parse.urlencode(data).encode('utf-8')
    
    result = opener.open(loginpage)  # without credentials, to get cookie
    
    result.read()
    
    for c in cookiejar:    # just debugging to see if we got a cookie
      print(c)
    
    
    result = opener.open(loginpage, binData)   # now send the crendentials
    
    page = result.read().decode()              # read() will read after redirects!
    
    f = open("out.html", "w")                  # store output in a file
    f.write(page)                              # if everything worked, this should look
    f.close()                                  # like the landing page
    

Please also note that most sites require **Javascript** , and Python can't
handle that. I suggest that you disable Javascript in your Browser, clear all
cookies from the site you try to access, and then carefully step through the
process using [Firebug](https://getfirebug.com/) and see which information
(cookies, GET/POST-parameters, redirects, etc.) gets passed around for a login
process. Then you can try to recreate this using python.

# Hotmail specific

Hotmail.com seems to be a suboptimal starting point to work with login forms.
If you look at the source (I highly suggest [Firebug](https://getfirebug.com/)
if you use Firefox), you can see that the "real" input field is not the
`<div>` that you seem to try to use, but this one:

    
    
    <input type="email" lang="en" name="login" id="i0116" maxlength="113"
     class="ltr_override" aria-labelledby="idLbl_PWD_Username">
    

and the password is contained in this one

    
    
    <input type="password" name="passwd" id="i0118" autocomplete="off"
     aria-labelledby="idDiv_PWD_PasswordExample">
    

If you enter dummy (or your real) information in the input fields and try to
login, you can see in Firebug's "Network" tab that some more information is
sent:

    
    
    LoginOptions    3
    NewUser 1
    PPFT    {very long encoded string}
    PPSX    Pas
    i1  0
    i12 1
    i13 0
    i14 226
    i15 2270
    i17 0
    i18 __Login_Strings|1,__Login_Core|1,
    i2  1
    i3  92877
    i4  0
    i7  0
    idsbho  1
    login   username123@example.com
    passwd  password456
    sso 0
    type    11
    

It is possible that `login` and `password` are sufficient, but since I don't
have a Hotmail account, I can't do further testing. Your general approach
using urllib.request seems to be OK (except that `geturl` does exactly that,
it gets an URL, so you should rename the variable html to something better).

