how to extract reviews from iframeurl returned by amazon api in python?
Question: I am trying to get the `text` content of reviews of a given product in
`amazon` using its `api`. But I am not able to work it out. Here is what I
have:

    
    
    result = api.item_lookup('B00062B6QY', ResponseGroup='Reviews',
         TruncateReviewsAt=256, IncludeReviewsSummary=False)
    iframeurl=result.xpath('//*[local-name()="IFrameURL"]/text()')[0].strip()
    print iframeurl
    reviews=requests.get(iframeurl)
    reviews.raise_for_status()
    #data = json.loads(reviews.text)
    root = ET.fromstring(reviews.text)
    print root
    

The output is:

    
    
    http://www.amazon.com/reviews/iframe?akid=helloworld&alinkCode=xm2&asin=B00062B6QY&atag=welcomehome-20&exp=2014-01-28T19%3A06%3A20Z&summary=0&truncate=256&v=2&sig=HIDDEN%3D
    Traceback (most recent call last):
      File "amazon_api_new.py", line 36, in <module>
        root = ET.fromstring(reviews.text)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1300, in XML
        parser.feed(text)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1642, in feed
        self._raiseerror(v)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
        raise err
    xml.etree.ElementTree.ParseError: mismatched tag: line 867, column 2
    

PS: I have changed the `iframeurl` printed out just to clear the `api key`
details

EDIT: image![enter image description here](http://i.stack.imgur.com/6SIa6.png)
from `firebug`


Answer: instead of using ElementTree, try to load `reviews.text` to
[lxml](http://lxml.de/parsing.html) like:

    
    
    >>> from lxml import etree
    >>> parser = etree.HTMLParser()
    >>> tree   = etree.parse(StringIO(reviews.text), parser)
    
    >>> result = etree.tostring(tree.getroot(),
    ...                         pretty_print=True, method="html")
    >>> print(result)
    ...
    

of course, you can then use [lxml xpath](http://lxml.de/xpathxslt.html) for
further parsing

