parse the stream with lxml like sofrom lxml html import parse from urllib import urlopen parsed parse(urlopen('doc parsed getroot(using this objectyou can extract all html tags of particular typesuch as table tags containing the data of interest as simple motivating examplesuppose you wanted to get list of every url linked to in the documentlinks are tags in html using the document root' findall method along with an xpath ( means of expressing "querieson the document)in [ ]links doc findall(// 'in [ ]links[ : out[ ][but these are objects representing html elementsto get the url and link text you have to use each element' get method (for the urland text_content method (for the display text)in [ ]lnk links[ in [ ]lnk out[ ]in [ ]lnk get('href'out[ ]'in [ ]lnk text_content(out[ ]'special editionsthusgetting list of all urls in the document is matter of writing this list comprehensionin [ ]urls [lnk get('href'for lnk in doc findall(// ')in [ ]urls[- :out[ ]['''''''reading and writing data in text format www it-ebooks info