def _get_pages(self, locations, req):
pending_queue = Queue()<EOL>for location in locations:<EOL><INDENT>pending_queue.put(location)<EOL><DEDENT>done = []<EOL>seen = set()<EOL>threads = []<EOL>for i in range(min(<NUM_LIT:10>, len(locations))):<EOL><INDENT>t = threading.Thread(target=self._get_queued_page, args=(req, pending_queue, done, seen))<EOL>t.setDaemon(True)<EOL>threads.append(t)<EOL>t.start()<EOL><DEDENT>for t in threads:<EOL><INDENT>t.join()<EOL><DEDENT>return done<EOL>
Yields (page, page_url) from the given locations, skipping
        locations that have errors, and adding download/homepage links