title: Scroll modal window using Selenium in Python
body: <p>I am trying to scrape links to song pages for some artists on genius.com, but I'm running into issues because the links to the individual song pages are displayed inside a popup modal window. </p>
  
  <p>The modal window doesn't load all links in one go, and instead loads more content via ajax when you scroll down to the bottom of the modal.</p>
  
  <p>I tried using code to scroll to the bottom of the page but unfortunately that just scrolled in the window behind the modal rather than the modal itself:</p>
  
  <pre><code>driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
  </code></pre>
  
  <p>So then I tried selecting the last element in the modal and scrolling to that (with the idea of doing that a few times until all song pages had been loaded), but it wouldn't scroll far enough to get the website to load more content</p>
  
  <pre><code>last_element = driver.find_elements_by_xpath('//div[@class="mini_card-metadata"]')[-1]
  last_element.location_once_scrolled_into_view
  </code></pre>
  
  <p>Here is my code so far:</p>
  
  <pre><code>import os
  from bs4 import BeautifulSoup
  from selenium import webdriver
  
  chrome_driver = "/Applications/chromedriver"
  os.environ["webdriver.chrome.driver"] = chrome_driver
  driver = webdriver.Chrome(chrome_driver)
  
  base_url = 'https://genius.com/artists/Stormzy'
  driver.get(base_url)
  
  xpath_str = '//div[contains(text(),"Show all songs by Stormzy")]'
  driver.find_element_by_xpath(xpath_str).click()
  </code></pre>
  
  <p>Is there a way to extract all the song page links for the artist?</p>