''nowfinding the right tables in the document can be matter of trial and errorsome websites make it easier by giving table of interest an id attribute determined that these were the two tables containing the call data and put datarespectivelytables doc findall(//table'calls tables[ puts tables[ each table has header row followed by each of the data rowsin [ ]rows calls findall(//tr'for the header as well as the data rowswe want to extract the text from each cellin the case of the header these are th cells and td cells for the datadef _unpack(rowkind='td')elts row findall(//%skindreturn [val text_content(for val in eltsthuswe obtainin [ ]_unpack(rows[ ]kind='th'out[ ]['strike''symbol''last''chg''bid''ask''vol''open int'in [ ]_unpack(rows[ ]kind='td'out[ ][' ''aapl '' ' '' '' '' '' 'nowit' matter of combining all of these steps together to convert this data into dataframe since the numerical data is still in string formatwe want to convert somebut perhaps not all of the columns to floating point format you could do this by handbutluckilypandas has class textparser that is used internally in the read_csv and other parsing functions to do the appropriate automatic type conversionfrom pandas io parsers import textparser def parse_options_data(table)rows table findall(//tr'header _unpack(rows[ ]kind='th'data [_unpack(rfor in rows[ :]return textparser(datanames=headerget_chunk(finallywe invoke this parsing function on the lxml table objects and get dataframe results data loadingstorageand file formats www it-ebooks info