Form submit using Python lxml
Question: I have a login page which is like this

    
    
    <form method="POST" name="DefaultForm" action="SOME_URL" onSubmit="return (isReady(this));" autocomplete="off">
    
    <input name="action" type="hidden" value="SOME_VALUE">
    <input name="serverTimeStamp" type="hidden" value="SOME_VALUE">
    <input name="clientTimeStamp" type="hidden" value="">
    <input name="clientIP" type="hidden" value="SOME_VALUE">
    
    <TABLE height="400" cellSpacing="0" cellPadding="0" width="540" align="center" background="images/bkground.gif" border="0">
    ...
    <INPUT class="inputStyle" type="Input"  name="username" size="20">
    <INPUT class="inputStyle" type="password"  maxLength="28" name="password" size="20">
    ...
    </TABLE>
    </form>
    

Using Python and lxml/Requests modules

    
    
    session_requests = requests.session()
    result = session_requests.get(url)
    tree = html.fromstring(result.content)
    if tree.find('form') :
    print "do something"
    else :
    print "do something else"
    

but I am not able to get this to work, the find() always returns **None**
always comes to else block. Can someone suggest a solution?


Answer: Why you see none is because the form is the _root_ element:

    
    
        h = """<form method="POST" name="DefaultForm" action="SOME_URL" onSubmit="return (isReady(this));" autocomplete="off">
    
    <input name="action" type="hidden" value="SOME_VALUE">
    <input name="serverTimeStamp" type="hidden" value="SOME_VALUE">
    <input name="clientTimeStamp" type="hidden" value="">
    <input name="clientIP" type="hidden" value="SOME_VALUE">
    
    <TABLE height="400" cellSpacing="0" cellPadding="0" width="540" align="center" background="images/bkground.gif" border="0">
    <INPUT class="inputStyle" type="Input"  name="username" size="20">
    <INPUT class="inputStyle" type="password"  maxLength="28" name="password" size="20">
    </TABLE>
    </form>"""
    
    x = html.fromstring(h)
    print(x.attrib)
    print(x)
    print(x.find("form"))
    

Just doing the above:

    
    
    {'action': 'SOME_URL', 'autocomplete': 'off', 'onsubmit': 'return (isReady(this));', 'method': 'POST', 'name': 'DefaultForm'}
    <Element form at 0x7f2b9c28eb50>
    None
    

If we wrap the form in a div:

    
    
    h = """<div>
          <form method="POST" name="DefaultForm" action="SOME_URL" onSubmit="return (isReady(this));" autocomplete="off">
    <input name="action" type="hidden" value="SOME_VALUE">
    <input name="serverTimeStamp" type="hidden" value="SOME_VALUE">
    <input name="clientTimeStamp" type="hidden" value="">
    <input name="clientIP" type="hidden" value="SOME_VALUE">
    <TABLE height="400" cellSpacing="0" cellPadding="0" width="540" align="center" background="images/bkground.gif" border="0">
    <INPUT class="inputStyle" type="Input"  name="username" size="20">
    <INPUT class="inputStyle" type="password"  maxLength="28" name="password" size="20">
    </TABLE>
    </form>
    </div>"""
    
    from lxml import html
    import lxml.etree as et
    x = html.fromstring(h)
    print x
    print(x.find("form"))
    

Now the div is the root and _find_ , finds the form:

    
    
    <Element div at 0x7f05966b3b50>
    <Element form at 0x7f0597a44ba8>
    

