Python Beautiful Soup Error : list index out of range
Question: I am getting the following error with the script, i want to do is,

if i get a correct URL, i want it to check with BeautifulSoup if there is a
form with value button "Send"

Traceback (most recent call last): File "tester.py", line 27, in if
viewstate[0]['value'] == "Send": IndexError: list index out of range

    
    
    #!/usr/bin/env python
    
    import urllib2
    import sys
    import os
    
    url = sys.argv[1]
    open_dir_list = open("dirlist.txt",'r')
    dirs = open_dir_list.read().split("\n")
    open_dir_list.close()
    
    for dir in dirs:
    
            uri = url+"/"+dir
    
            try:
                    from BeautifulSoup import BeautifulStoneSoup
                    response = urllib2.urlopen(uri)
                    if response.getcode() == 200:
                            s = uri
                            soup = BeautifulStoneSoup(uri)
                            viewstate = soup.findAll("input", {"type": "submit"})
                            if viewstate[0]['value'] == "Send":
                                    print("Yay!!!")
                            else:
                                    print("There's nothing here")
            except urllib2.HTTPError, e:
                    if e.code == 401:
                            print "[!] Authorization Required %s " % (uri)
                    elif e.code == 403:
                            print "[!] Forbidden %s " % (uri)
                    elif e.code == 404:
                            print "[-] Not Found %s " % (uri)
                    elif e.code == 503:
                            print "[!] Service Unavailable %s " % (uri)
                    else:
                            print "[?] Unknwon"
    
    
    
    print "\n:. FINISH :.\n"
    

It is working fine with this script, but it only checks only a given path

    
    
    import urllib
    f = urllib.urlopen("http://xxx.xxx.xxx.xxx/button.jsp")
    s = f.read()
    f.close()
    
    from BeautifulSoup import BeautifulStoneSoup
    soup = BeautifulStoneSoup(s)
    
    viewstate = soup.findAll("input", {"type": "submit"})
    if viewstate[0]['value'] == "Send":
            print(" Yay!!!")
    else:
            print("No Submit Button")
    


Answer: Apart from what I mentioned in a comment you are not passing the returned html
you are passing `uri = url+"/"+dir` instead of `response.read()` to
`BeautifulSoup` so you are searching for the tag in `uri` which I imagine
certainly does not contain any tags. You need to pass `read` as below:

    
    
         response = urllib2.urlopen(uri)
                if response.getcode() == 200:                       
                        soup = BeautifulStoneSoup(response.read())
    

If you want the first match use `.find` using `if viewstate` to make sure it
matched something, you can also iterate over the file object getting a line at
a time:

    
    
    from BeautifulSoup import BeautifulStoneSoup
    import urllib2
    import sys
    
    url = sys.argv[1]
    with  open("dirlist.txt",'r') as f:
        for dir in f:
            uri = url + "/" + dir.rstrip()
            try:
                response = urllib2.urlopen(uri)
                if response.getcode() == 200:
                        soup = BeautifulStoneSoup(response.read())
                        viewstate = soup.find("input", {"type": "submit"})
                        if viewstate and viewstate.get("value") == "Send":
                            print("Shell is found!! Yay!!!")
                        else:
                            print("There's nothing here")
            except urllib2.HTTPError, e:
                    if e.code == 401:
                        print "[!] Authorization Required %s " % (uri)
                    elif e.code == 403:
                        print "[!] Forbidden %s " % (uri)
                    elif e.code == 404:
                        print "[-] Not Found %s " % (uri)
                    elif e.code == 503:
                        print "[!] Service Unavailable %s " % (uri)
                    else:
                        print "[?] Unknwon"
    
    
    
        print "\n:. FINISH :.\n"
    

