Extract Meta Keywords From Webpage?
Question: I need to extract the meta keywords from a web page using Python. I was
thinking that this could be done using urllib or urllib2, but I'm not sure.
Anyone have any ideas?

I am using Python 2.6 on Windows XP


Answer: [lxml](http://codespeak.net/lxml/index.html#documentation) is faster than
BeautifulSoup (I think) and has much better functionality, while remaining
relatively easy to use. Example:

    
    
    52> from urllib import urlopen
    53> from lxml import etree
    
    54> f = urlopen( "http://www.google.com" ).read()
    55> tree = etree.HTML( f )
    61> m = tree.xpath( "//meta" )
    
    62> for i in m:
    ..>     print etree.tostring( i )
    ..>
    <meta http-equiv="content-type" content="text/html; charset=ISO-8859-2"/>  
    

Edit: another example.

    
    
    75> f = urlopen( "http://www.w3schools.com/XPath/xpath_syntax.asp" ).read()
    76> tree = etree.HTML( f )
    85> tree.xpath( "//meta[@name='Keywords']" )[0].get("content")
    85> "xml,tutorial,html,dhtml,css,xsl,xhtml,javascript,asp,ado,vbscript,dom,sql,colors,soap,php,authoring,programming,training,learning,b
    eginner's guide,primer,lessons,school,howto,reference,examples,samples,source code,tags,demos,tips,links,FAQ,tag list,forms,frames,color table,w3c,cascading
     style sheets,active server pages,dynamic html,internet,database,development,Web building,Webmaster,html guide"
    

BTW: [XPath](http://www.w3schools.com/XPath/xpath_syntax.asp) is worth
knowing.

Another edit:

Alternatively, you can just use regexp:

    
    
    87> f = urlopen( "http://www.w3schools.com/XPath/xpath_syntax.asp" ).read()
    88> import re
    101> re.search( "<meta name=\"Keywords\".*?content=\"([^\"]*)\"", f ).group( 1 )
    101>"xml,tutorial,html,dhtml,css,xsl,xhtml,javascript,asp,ado,vbscript,dom,sql, ...etc...
    

...but I find it less readable and more error prone (but involves only
standard module and still fits on one line).

