Python & BS4: Exclude blank and Grand Total Row
Question: Below is my code to extract data out of an HTML document and place it into
variables. I need to exclude the blank lines, as well as the "grand total"
line. I've added the HTML input of those segments beneath my code. I'm not
sure how to make it work. I can't use `len()` because the length is variable.
Any help?

    
    
    from bs4 import BeautifulSoup
    import urllib
    import re
    import HTMLParser
    html = urllib.urlopen('RanpakAllocations.html').read()
    parser = HTMLParser.HTMLParser()
    #unescape doesn't seem to work
    output = parser.unescape(html)
    
    soup1 = BeautifulSoup(output, "html.parser")
    Customer_No = []
    Serial_No = []
    data = []
    #for hit in soup.findAll(attrs={'class' : 'MYCLASS'}):
    rows = soup1.find_all("tr")
    title = rows[0]
    headers = rows[1]
    datarows = rows[2:]
    
    fields = []
    
    try :
        for row in datarows :
            find_data = row.find_all(attrs={'face' : 'Arial,Helvetica,sans-serif'})
            count = 0
            for hit in find_data:
                data = hit.text
                count = count + 1
                if count == 3 :
                    CSNO = data
                if count == 9 :
                    ITNO = data
                else :
                    continue
    
            print CSNO, ITNO
            print "new row"
    except:
        pass
    

Here is the input. The first `<tr>` is my last row of data, however my loop is
repeating for the blank rows and the grand total row below it.

    
    
    <tr>
            <td nowrap="nowrap" align="left"><font size="3" face="Arial,Helvetica,sans-serif">12</font></td>
            <td nowrap="nowrap" align="left"><font size="3" face="Arial,Helvetica,sans-serif">F5684</font></td>
            <td nowrap="nowrap" align="left"><font size="3" face="Arial,Helvetica,sans-serif">20182</font></td>
            <td nowrap="nowrap" align="left"><font size="3" face="Arial,Helvetica,sans-serif">VELOCITY SOLUTIONS INC.</font></td>
            <td nowrap="nowrap" align="left"><font size="3" face="Arial,Helvetica,sans-serif">EQPRAN77717</font></td>
            <td nowrap="nowrap" align="left"><font size="3" face="Arial,Helvetica,sans-serif">RANPAK FILLPAK TT 2</font></td>
            <td nowrap="nowrap" align="left"><font size="3" face="Arial,Helvetica,sans-serif">W/UNIVERSAL STAND S/N 51345563</font></td>
            <td nowrap="nowrap" align="right"><font size="3" face="Arial,Helvetica,sans-serif">1</font></td>
            <td nowrap="nowrap" align="left"><font size="3" face="Arial,Helvetica,sans-serif">51345563</font></td>
          </tr>
          <tr>
            <td nowrap="nowrap" align="left"><font size="1">&nbsp;</font></td>
            <td nowrap="nowrap" align="left"><font size="1">&nbsp;</font></td>
            <td nowrap="nowrap" align="left"><font size="1">&nbsp;</font></td>
            <td nowrap="nowrap" align="left"><font size="1">&nbsp;</font></td>
            <td align="left" colspan="5"><font size="1">&nbsp;</font></td>
          </tr>
          <tr>
            <td align="left"><font size="3" face="Arial,Helvetica,sans-serif">&nbsp;</font></td>
            <td align="left"><font size="3" face="Arial,Helvetica,sans-serif">Grand Total</font></td>
            <td align="left" colspan="7"><font size="1">&nbsp;</font></td>
          </tr>
          <tr>
            <td>&nbsp;</td>
            <td>&nbsp;</td>
            <td>&nbsp;</td>
            <td>&nbsp;</td>
            <td>&nbsp;</td>
            <td>&nbsp;</td>
            <td>&nbsp;</td>
            <td>&nbsp;</td>
            <td>&nbsp;</td>
          </tr> 
    


Answer: I would do something like this:

    
    
    from bs4 import BeautifulSoup
    
    content = '''
    <root>
        <tr>
            <td nowrap="nowrap" align="left"><font size="3" face="Arial,Helvetica,sans-serif">12</font></td>
            <td nowrap="nowrap" align="left"><font size="3" face="Arial,Helvetica,sans-serif">F5684</font></td>
            <td nowrap="nowrap" align="left"><font size="3" face="Arial,Helvetica,sans-serif">20182</font></td>
            <td nowrap="nowrap" align="left"><font size="3" face="Arial,Helvetica,sans-serif">VELOCITY SOLUTIONS INC.</font></td>
            <td nowrap="nowrap" align="left"><font size="3" face="Arial,Helvetica,sans-serif">EQPRAN77717</font></td>
            <td nowrap="nowrap" align="left"><font size="3" face="Arial,Helvetica,sans-serif">RANPAK FILLPAK TT 2</font></td>
            <td nowrap="nowrap" align="left"><font size="3" face="Arial,Helvetica,sans-serif">W/UNIVERSAL STAND S/N 51345563</font></td>
            <td nowrap="nowrap" align="right"><font size="3" face="Arial,Helvetica,sans-serif">1</font></td>
            <td nowrap="nowrap" align="left"><font size="3" face="Arial,Helvetica,sans-serif">51345563</font></td>
        </tr>
        <tr>
            <td nowrap="nowrap" align="left"><font size="1">&nbsp;</font></td>
            <td nowrap="nowrap" align="left"><font size="1">&nbsp;</font></td>
            <td nowrap="nowrap" align="left"><font size="1">&nbsp;</font></td>
            <td nowrap="nowrap" align="left"><font size="1">&nbsp;</font></td>
            <td align="left" colspan="5"><font size="1">&nbsp;</font></td>
        </tr>
        <tr>
            <td align="left"><font size="3" face="Arial,Helvetica,sans-serif">&nbsp;</font></td>
            <td align="left"><font size="3" face="Arial,Helvetica,sans-serif">Grand Total</font></td>
            <td align="left" colspan="7"><font size="1">&nbsp;</font></td>
        </tr>
        <tr>
            <td>&nbsp;</td>
            <td>&nbsp;</td>
            <td>&nbsp;</td>
            <td>&nbsp;</td>
            <td>&nbsp;</td>
            <td>&nbsp;</td>
            <td>&nbsp;</td>
            <td>&nbsp;</td>
            <td>&nbsp;</td>
        </tr>
    </root>'''
    
    soup = BeautifulSoup(content, 'html')
    
    answer = []
    rows = soup.find_all('tr')
    
    for row in rows:
        if not row.text.strip():
            continue
    
        row_text = []
        for cell in row.find_all('td'):
            if cell.text.strip():
                row_text.append(cell.text)
    
        answer.append(row_text)
    
    print(answer)
    

**Output**

    
    
    [[u'12', u'F5684', u'20182', u'VELOCITY SOLUTIONS INC.', u'EQPRAN77717', u'RANPAK FILLPAK TT 2', u'W/UNIVERSAL STAND S/N 51345563', u'1', u'51345563'], [u'Grand Total']]
    

You can skip over entire empty rows using `if not row.text.strip(): continue`
(`row.text.strip()` returns an empty string, which evaluates to `False`).

For rows that you do iterate over, you can check each cell is not empty using
`if cell.text.strip()` before saving the relevant text.

