python mechanize submitting form kicks me back to root
Question: i have a mechanize python script written for submitting forms to inquire drug
information. and when i run it, it gives me no error message, but when i look
at the response, it's not what I see on my browser view-source page. i checked
the urls after the submission:

here's what I got:

<http://www.accessdata.fda.gov/scripts/cder/drugsatfda/index.cfm>

here's addresses I'm supposed to get:

<http://www.accessdata.fda.gov/scripts/cder/drugsatfda/index.cfm?fuseaction=Search.DrugDetails>

I see that the second url does not contain my query text, does that mean i
need cookies? if so, how?

this is my code snippet:

    
    
    br = mechanize.Browser()
    cj = cookielib.LWPCookieJar()
    br.set_cookiejar(cj)
    ....
    br.addheaders = [('User-agent', 'Mozilla/6.0 (X11; U; i686; en-US; rv:1.9.0.1) Gecko/2008071615 OS X 10.2 Firefox/3.0.1')]
    
    fda_url2 = 'http://www.accessdata.fda.gov/scripts/cder/drugsatfda/index.cfm?fuseaction=Search.Addlsearch_drug_name'
    
    print br.open(fda_url2).geturl()
    
    
    for f in br.forms():
       print 'this is a form'
       print f
    
    br.select_form('searchoptionB')
    br.form['ApplNo'] = '018780'
    html = br.submit(name = 'Search_Button')
    
    print html.geturl()
    

the print form output was:

    
    
    <searchoptionB POST http://www.accessdata.fda.gov/scripts/cder/drugsatfda/index.cfm application/x-www-form-urlencoded
      <HiddenControl(fuseaction=Search.SearchAction) (readonly)>
      <HiddenControl(SearchType=AddlSearch) (readonly)>
      <HiddenControl(SearchOption=B) (readonly)>
      <TextControl(ApplNo=)>
      <SubmitControl(Search_Button=Submit) (readonly)>
      <SubmitControl(clearcriteria=Clear) (readonly)>>
    

sorry for the long post ;p


Answer: **UPD** Regarding your comment. Here's my test file:

    
    
    #!/usr/bin/env python
    import mechanize
    import cookielib
    
    br = mechanize.Browser()
    cj = cookielib.LWPCookieJar()
    br.set_cookiejar(cj)
    br.addheaders = [('User-agent', 'Mozilla/6.0 (X11; U; i686; en-US; rv:1.9.0.1) Gecko/2008071615 OS X 10.2 Firefox/3.0.1')]
    
    fda_url2 = 'http://www.accessdata.fda.gov/scripts/cder/drugsatfda/index.cfm?fuseaction=Search.Addlsearch_drug_name'
    
    print br.open(fda_url2).geturl()
    
    
    for f in br.forms():
       print 'this is a form'
       print f
    
    br.select_form('searchoptionB')
    br.form['ApplNo'] = '018780'
    html = br.submit()
    
    print br.response().read()
    

And this is what I get when running it:

    
    
    $ ./test.py  | grep HUM
                <td valign="top" class="product_table" height="24"><a href="index.cfm?fuseaction=Search.Overview&DrugName=HUMULIN%20R">
     HUMULIN R
     INSULIN RECOMBINANT HUMAN
                <td valign="top" class="product_table" height="24"><a href="index.cfm?fuseaction=Search.Overview&DrugName=HUMULIN%20R%20PEN">
     HUMULIN R PEN
     INSULIN RECOMBINANT HUMAN
    

Maybe you've somehow changed your script or query a different page?

* * *

Have you tried to submit this form in a browser?

If I navigate to [this
URL](http://www.accessdata.fda.gov/scripts/cder/drugsatfda/index.cfm?fuseaction=Search.Addlsearch_drug_name),
fill in "Option B" with "018780" and hit "Submit", the browser indeed
redirects me to
<http://www.accessdata.fda.gov/scripts/cder/drugsatfda/index.cfm>, which
contains the search results.

Try adding this to the end of your code snippet:

    
    
    print br.response().read()
    

This will output the HTML of the page, and it does contain search results as
expected.

> I see that the second url does not contain my query text, does that mean i
> need cookies? if so, how?

This form is sent via POST, and in this case all parameters are embedded in
request body (RFC 2616).

