title: Scrape spotify web interface
body: <p>I'm trying to get the number of plays for the top songs from a number of artists on Spotify using python and splinter. </p>
  
  <p>If you fill in the username and password below with yours, you should be able to run the code.</p>
  
  <pre><code>from splinter import Browser
  import time
  from bs4 import BeautifulSoup
  
  browser = Browser()
  url = 'http://play.spotify.com'
  browser.visit(url)
  time.sleep(2)
  button = browser.find_by_id('has-account')
  button.click()
  time.sleep(1)
  browser.fill('username', 'your_username')
  browser.fill('password', 'your_password')
  buttons = browser.find_by_css('button')
  visible_buttons = [button for button in buttons if button.visible]
  login_button = visible_buttons[-1]
  login_button.click()
  time.sleep(1)
  browser.visit('https://play.spotify.com/artist/5YGY8feqx7naU7z4HrwZM6')
  time.sleep(10)
  </code></pre>
  
  <p>So far, so good. If you open up firefox, you'll can see Miley Cyrus's artist page, including the number of plays for top tracks. </p>
  
  <p>If you open up the Firefox Developer Tools Inspector and hover, you can see the name of the song in <code>.tl-highlight</code> elements, and the number of plays in <code>.tl-listen-count</code> elements. However, I've found it impossible (at least on my machine) to access these elements using <code>splinter</code>. Moreover, when I try to get the source for the entire page, the elements that I can see by hovering my mouse over them in Firefox don't show up in what is ostensibly the page source.</p>
  
  <pre><code>html = browser.html
  soup = BeautifulSoup(html)
  output = soup.prettify()
  with open('miley_cyrus_artist_page.html', 'w') as output_f:
      output_f.write(output)
  browser.quit()
  </code></pre>
  
  <p>I don't think I know enough about web programming to know what the issue is here--Firefox sees all the DOM elements clearly, but splinter that is driving Firefox does not.</p>