Modifying html files in terms of tags?
Question: I have several html files, whose contents are like this:

    
    
    <html>
        <header>
            <title>A test</title>
        </header>
        <body>
            <table>
                <tr>
                    <td id="MenuTD" style="vertical-align: top;"> 
                        Stuff here <a>with a link</a>
                        <p>Or paragraph tags</p>
                        <div>Or a DIV</div>
                    </td>
                    <td>Another TD element, without the MenuTD id</td>
                </tr>
            </table>
            <div>
                 <link rel="stylesheet" href="\d\d\d\d_files/zannotationtargettoggle.css" type="text/css">
            </div>
        </body>
    </html>
    

where `\d` is a placeholder of a digit, and the exact digit varies from file
to file.

I want to write a Python program to convert each html file to the following
form:

    
    
    <html>
        <header>
            <title>A test</title>
        </header>
        <body>
            <link rel="stylesheet" href="\d\d\d\d_files/zannotationtargettoggle.css" type="text/css">
            <td id="MenuTD" style="vertical-align: top;"> 
                Stuff here <a>with a link</a>
                <p>Or paragraph tags</p>
                <div>Or a DIV</div>
            </td>
        </body>
    </html>
    

Specifically,

  1. How can we extract the header tag `<header>...</header>` and `<link rel="stylesheet" href="\d\d\d\d_files/zannotationtargettoggle.css" type="text/css">` , given that they don't have IDs? 

  2. If the body tag has attributes e.g. `<body style="margin-left: 6px; cursor: default;" onload="InitBody();">...</body>`, How shall we first empty the content `...` inside its beginning and ending tags, and then add `<link rel="stylesheet" href="\d\d\d\d_files/zannotationtargettoggle.css" type="text/css">` and the content of `menu_td` inside?

Thanks!


Answer: You can use [BeautifulSoup](http://www.crummy.com/software/BeautifulSoup/) to
modify the input document:

    
    
    import bs4
    
    doc = bs4.BeautifulSoup(s) # s your input html
    td = doc.find('td')
    doc.find('table').replace_with(doc.find('link'))
    doc.find('div').replace_with(td)
    

Testing the resulting document:

    
    
    >>> print str(doc)
    <html>
    <body><header>
    <title>A test</title>
    </header>
    <link href="\d\d\d\d_files/zannotationtargettoggle.css" rel="stylesheet" type="text/css"/>
    <td id="MenuTD" style="vertical-align: top;"> 
                        Stuff here <a>with a link</a>
    <p>Or paragraph tags</p>
    <div>Or a DIV</div>
    </td>
    </body></html>
    

or you could construct a new document:

    
    
    doc = bs4.BeautifulSoup(s)
    doc2 = bs4.BeautifulSoup('<html />')
    doc2.html.append(doc.header)
    doc2.html.append(doc2.new_tag('body'))
    doc2.body.append(doc.link)
    doc2.body.append(doc.find('td'))
    

