title: Grubhub scraping: How to extract the full html
body: <p>I am scraping grubhub and I am not able to scrape the full menu.</p>
  <p><a href="https://www.grubhub.com/restaurant/buca-di-beppo-1875-s-bascom-ave-campbell/335944" rel="nofollow noreferrer">https://www.grubhub.com/restaurant/buca-di-beppo-1875-s-bascom-ave-campbell/335944</a></p>
  <p>For example in the above,it only scrapes appitizers. Scrolling is required to get the rest, however the captcha realizes it is automated (with selenium) and I cannot scrape anymore.</p>
  <p>Here is what I have:</p>
  <pre><code>driver.get(link)
  time.sleep(2)
  page = driver.page_source
  soup = BeautifulSoup(page, 'html.parser')
  dishes = soup.find_all('div', class_='menuItemNew-name')
  descs = soup.find_all('div', class_='padding-y-2')
  dishes_ = []
  descs_ = []
  for items in dishes:
      dishes_ += items.find_all(text=True)
  for items in descs:
      descs_ += items.find_all(text=True)
  
  print(dishes_)
  print(descs_)
  </code></pre>
  <p><code>descs</code> are the descirptions of each dish which I also want to scrape.</p>
  <p>How do I get the full menu (and the google maps link at the very bottom of the page if possible?)</p>