title: Selenium: retrieve data that loads while scrolling down
body: <p>I'm trying to retrieve elements in a page that has an ajax-load scroll-down functionality alla Twitter. For some reason this isn't working properly. I added some print statements to debug it and I always get the same amount of items and then the function returns. What am I doing wrong here?</p>
  
  <pre><code>wd = webdriver.Firefox()
  wd.implicitly_wait(3)
  
  def get_items(items):
      print len(items)
      wd.execute_script("window.scrollTo(0, document.body.scrollHeight);")
      # len(items) and len(wd.find_elements-by...()) both always seem to return the same number
      # if I were to start the loop with while True: it would work, but of course... never end
      while len(wd.find_elements_by_class_name('stream-item')) &gt; len(items):
          items = wd.find_elements_by_class_name('stream-item')
          print items
          wd.execute_script("window.scrollTo(0, document.body.scrollHeight);")
      return items
  
  def test():
      get_page('http://twitter.com/')
      get_items(wd.find_elements_by_class_name('stream-item'))
  </code></pre>