Find all attributes for each HTML tag This code outputs each tag followed by its attributes in the given HTML, both in lexicographic order. It must be possible to keep the spirit of the code without these many nested cycles and conditionals.

from bs4 import BeautifulSoup

htmlist = ['&lt;div class="portal" role="navigation" id=\'p-coll-print_export\'&gt;',
     '&lt;h3&gt;Print/export&lt;/h3&gt;',
     '&lt;div class="body"&gt;',
     '&lt;ul&gt;',
     '&lt;li id="coll-create_a_book"&gt;&lt;a href="/w/index.php?title=Special:Book&amp;amp;bookcmd=book_creator&amp;amp;referer=Main+Page"&gt;Create a book&lt;/a&gt;&lt;/li&gt;',
     '&lt;li id="coll-download-as-rl"&gt;&lt;a href="/w/index.php?title=Special:Book&amp;amp;bookcmd=render_article&amp;amp;arttitle=Main+Page&amp;amp;oldid=560327612',
     '&lt;li id="t-print"&gt;&lt;a href="/w/index.php?title=Main_Page&amp;amp;printable=yes" title="Printable version of this page [p]" accesskey="p"&gt;Printable version&lt;/a&gt;&lt;/li&gt;',
     '&lt;/ul&gt;',
     '&lt;/div&gt;',
     '&lt;/div&gt;']

soup = BeautifulSoup("\n".join(htmlist), "html.parser")

tags = sorted(set([tag.name for tag in soup.find_all()]))

for tag in tags:
    tags_keys = list()
    for line in htmlist:
        aux=BeautifulSoup(line, "html.parser").find(tag)
        if aux:
            tags_keys.extend(list(aux.attrs))
    print(tag+":"+",".join(sorted(set(tags_keys))))


I tried something along the lines of what's below (in place of the outermost for), but couldn't even get it to work.

for tag in tags:
    print(tag+":"+",".join(sorted(soup.find(tag).attrs.keys())))




Edit: There was a typo in the code, htmlist was supposed to be as follows.

htmlist = ['&lt;div class="portal" role="navigation" id="p-coll-print_export"&gt;',\
           '&lt;h3&gt;Print/export&lt;/h3&gt;',\
           '&lt;div class="body"&gt;',\
           '&lt;ul&gt;',\
           '&lt;li id="coll-create_a_book"&gt;&lt;a href="/w/index.php?title=Special:Book&amp;amp;bookcmd=book_creator&amp;amp;referer=Main+Page"&gt;Create a book&lt;/a&gt;&lt;/li&gt;',\
           '&lt;li id="coll-download-as-rl"&gt;&lt;a href="/w/index.php?title=Special:Book&amp;amp;bookcmd=render_article&amp;amp;arttitle=Main+Page&amp;amp;oldid=560327612&amp;amp;writer=rl"&gt;Download as PDF&lt;/a&gt;&lt;/li&gt;',\
           '&lt;li id="t-print"&gt;&lt;a href="/w/index.php?title=Main_Page&amp;amp;printable=yes" title="Printable version of this page [p]" accesskey="p"&gt;Printable version&lt;/a&gt;&lt;/li&gt;',\
           '&lt;/ul&gt;',\
           '&lt;/div&gt;',\
           '&lt;/div&gt;',\
          ]
Notice in @alecxe's answer how he flipped the loops:
instead of iterating over tags and then lines,
he's iterating over lines and then tags.
This performs much better,
because only one BeautifulSoup is created per line,
in contrast with your implementation
where an instance is created for every tag and line pair,
which is many more instances of BeautifulSoup,
and wasted processing.

One more improvement over that could be using just one instance of BeautifulSoup,
if you can join all the lines into a single document,
instead of processing line by line:

soup = BeautifulSoup("\n".join(htmlist), "html.parser")

tags = defaultdict(set)
for tag in soup():
    tags[tag.name] |= set(tag.attrs)

for tag_name in sorted(tags):
    print("{name}:{attrs}".format(name=tag_name, attrs=",".join(sorted(tags[tag_name]))))


However this is not possible with the given lines,
because this line is invalid HTML and will mess up the document:


 '&lt;li id="coll-download-as-rl"&gt;&lt;a href="/w/index.php?title=Special:Book&amp;amp;bookcmd=render_article&amp;amp;arttitle=Main+Page&amp;amp;oldid=560327612',



If possible, I suggest to fix this line, and use the above approach.

 '&lt;li id="coll-download-as-rl"&gt;&lt;a href="/w/index.php?title=Special:Book&amp;amp;bookcmd=render_article&amp;amp;arttitle=Main+Page&amp;amp;oldid=560327612"&gt;&lt;/a&gt;&lt;/li&gt;',