Parsing html in Python 2.7 with regex - don't really understand that
Question: Sorry for being kinda dumb, but I really need help in Python.

    
    
    ['<a href="needs to be cut out">Foo to BAR</a>', '<a href="this also needs to be cut out">BAR to Foo</a>']
    

So I have this tuple, and I need to cut out what's inside that href attribute
and what's inside `<a>` tag - basically, I want to get a tuple that looks
like:

    
    
    [["needs to be cut out", "Foo to BAR"], ["this also needs to be cut out", "BAR to Foo"]]
    

inside href attribute there are a lot of special symbols for example,

    
    
    <a href="?a=p.stops&amp;direction_id=23600&amp;interval=1&amp;t=wml&amp;l=en">
    

As I think, there's too much trouble in using HTML parsers if I really don't
need to try to parse the object tree but only need a few url's and words from
the webpage. But I can't really understand how to form regexes. Regexes that I
formed seem to be completely wrong. So I'm asking if somebody could help me
with it.


Answer: Just use a HTML parser anyway. Python comes with a few included, and the
[`xml.etree.ElementTree`
API](http://docs.python.org/2/library/xml.etree.elementtree.html) is easier to
get working than a regular expression for even simple `<a>` tags with
arbitrary attributes:

    
    
    from xml.etree import ElementTree as ET
    
    texts = []
    for linktext in linkslist:
        link = ET.fromstring(linktext)
        texts.append([link.attrib['href'], link.text])
    

If you use `' '.join(link.itertext())` you can get the text out of _anything_
nested under the `<a>` tag, if you find that some of the links have nested
`<span>`, `<b>`, `<i>` or other inline tags to mark up the link text further:

    
    
    for linktext in linkslist:
        link = ET.fromstring(linktext)
        texts.append([link.attrib['href'], ' '.join(link.itertext())])
    

This gives:

    
    
    >>> from xml.etree import ElementTree as ET
    >>> linkslist = ['<a href="needs to be cut out">Foo to BAR</a>', '<a href="this also needs to be cut out">BAR to Foo</a>']     
    >>> texts = []
    >>> for linktext in linkslist:
    ...     link = ET.fromstring(linktext)
    ...     texts.append([link.attrib['href'], ' '.join(link.itertext())])
    ... 
    >>> texts
    [['needs to be cut out', 'Foo to BAR'], ['this also needs to be cut out', 'BAR to Foo']]
    

