title: Selenium Webdriver is not scrolling to click the load more button
body: <p>I am trying to build a verge archives scraper for news headlines, my primary aim is to scrape data from a given month and year. The code was working some days back and it was scrolling properly but now it is unable to scroll and it is getting stuck every time. I am trying to scroll by the Action chain of CTRL+END but it does not work. I tried other ways too, but no luck</p>
  <pre><code>def scrolling_func(wait,driver):
  print(&quot;It is trying to scroll&quot;)
  SCROLL_PAUSE_TIME = 5
  
  last_height = driver.execute_script(&quot;return document.body.scrollHeight&quot;)
  
  while True:
      ActionChains(driver).key_down(Keys.CONTROL).send_keys('END').key_up(Keys.CONTROL).perform()
      load_button = driver.find_element_by_css_selector('.p-button')
      # driver.execute_script(&quot;arguments[0].scrollIntoView();&quot;, load_button)
      element = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, '.p-button')))
      # ActionChains(driver).move_to_element(load_button).click().perform()  
      load_button.click()   
      time.sleep(SCROLL_PAUSE_TIME)
      new_height = driver.execute_script(&quot;return document.body.scrollHeight&quot;)
      if new_height == last_height:
          break
      last_height = new_height
  # driver.delete_all_cookies()
  time.sleep(1)
  ActionChains(driver).key_down(Keys.CONTROL).send_keys('HOME').key_up(Keys.CONTROL).perform()
  </code></pre>
  <p>And the scraper is</p>
  <pre><code>def scraper(years,months):
  PATH = r&quot;C:\Users\astar\Stock market tutorials\chromedriver_win64\chromedriver.exe&quot;
  options = webdriver.ChromeOptions()
  options.use_chromium = True
  
  
  options.add_argument('--ignore-certificate-errors')
  options.add_argument('--ignore-ssl-errors')
  driver = webdriver.Chrome(executable_path=PATH,options=options)
  
  driver.maximize_window()
  
  
  urls = parse_dates(years,months)
  
  final_headlines = []
  final_dates = []
  final_links = []
  
  for url in urls:
      driver.get(url)
      done=True
      while done:
          try:
              wait = WebDriverWait(driver,10)
              scrolling_func(wait,driver)
          except:
              done=False
      ActionChains(driver).key_down(Keys.CONTROL).send_keys('HOME').key_up(Keys.CONTROL).perform()
      soup = BeautifulSoup(driver.page_source,'lxml')
      #https://stackoverflow.com/questions/5041008/how-to-find-elements-by-class
      #https://stackoverflow.com/questions/42732958/python-parallel-execution-with-selenium
      #https://stackoverflow.com/questions/44245451/how-to-scrape-multiple-html-page-in-parallel-with-beautifulsoup-in-python
      #https://stackoverflow.com/questions/45816619/selenium-firefox-webdriver-for-python-keyerror-value
      num_articles = soup.find(&quot;h1&quot;,class_=&quot;p-page-title&quot;).text
      current = num_articles[num_articles.find(&quot;for&quot;)+4:num_articles.find(&quot;(&quot;)]
      articles_num = num_articles[num_articles.find(&quot;(&quot;)+1:-1]
      titles = soup.find_all(&quot;h2&quot;,class_=&quot;c-entry-box--compact__title&quot;)
      dates = soup.find_all(&quot;time&quot;,class_=&quot;c-byline__item&quot;)
  
      if articles_num != len(titles):
          logger.warning(&quot;Actual #articles {} and #scraped articles {} for {}&quot;.format(articles_num,len(titles),current))
      print(len(titles),len(dates))
  
      headlines_results = map(title_extractor,titles)
      dates_results = map(date_extractor,dates)
      links_results = map(link_extractor,titles)
  
      
      def list_process(gens):
          return [gen for gen in gens]
      
      headlines = list_process(headlines_results)
      dates = list_process(dates_results)
      links = list_process(links_results)
  
      final_headlines.extend(headlines) 
      final_dates.extend(dates) 
      final_links.extend(links)
  
      time.sleep(15)
  
  print(len(final_headlines),len(final_dates),len(final_links))    
  
  assert len(final_headlines)==len(final_dates)==len(final_links), f'Different lengths of headlines {len(headlines)} and date {len(dates)}'
  data = {&quot;Headlines&quot;:final_headlines,&quot;Dates&quot;:final_dates,&quot;Links&quot;:final_links}
  df = pd.DataFrame(data) 
  df.to_csv('file1.csv') 
  return df 
  </code></pre>
  <p>if <strong>name</strong> == &quot;<strong>main</strong>&quot;:
  scraper([&quot;2021&quot;],[&quot;3&quot;])</p>
  <p>As I said, it is unable to scroll, it was working well some days back but now it is breaking. Also earlier I had the issue of unable to load the entire list of the page as it was getting stuck. Can anyone help me here? Thanks in advance.</p>