even though it collected links to external pagesit didn' go off collecting links from any of the external pages we linked to this is great little program if we want to collect all the links in site but it doesn' give me all the information might need to build site mapit tells me which pages havebut it doesn' tell me which pages link to other pages if we want to do that insteadwe're going to have to make some modifications the first thing we should do is look at our data structures the set of collected links doesn' work anymorewe want to know which links were linked to from which pages the first thing we could dothenis turn that set into dictionary of sets for each page we visit the dictionary keys will represent the exact same data that is currently in the set the values will be sets of all the links on that page here are the changesfrom urllib request import urlopen from urllib parse import urlparse import re import sys link_regex re compile"]*href=['\"]([^'\"]+)['\"][^>]*>"class linkcollectordef __init__(selfurl)self url "self collected_links {self visited_links set(def collect_links(selfpath="/")full_url self url path self visited_links add(full_urlpage str(urlopen(full_urlread()links link_regex findall(pagelinks {self normalize_url(pathlink for link in linksself collected_links[full_urllinks for link in linksself collected_links setdefault(linkset()unvisited_links links differenceself visited_linksfor link in unvisited_linksif link startswith(self url)self collect_links(urlparse(linkpathdef normalize_url(selfpathlink)if link startswith("