title: Data scraping from Vivino.com
body: <p>I am trying to collect data from vivino.com and the DataFrame comes out empty, I can see that my soup is collecting the website info, but can't see where my error is.</p>
  <p>My code:</p>
  <pre><code>def get_data():  
  
      headers = {&quot;User-Agent&quot;:&quot;Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0&quot;, &quot;Accept-Encoding&quot;:&quot;gzip, deflate&quot;, &quot;Accept&quot;:&quot;text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8&quot;, &quot;DNT&quot;:&quot;1&quot;,&quot;Connection&quot;:&quot;close&quot;, &quot;Upgrade-Insecure-Requests&quot;:&quot;1&quot;}
  
      r = requests.get(&quot;https://www.vivino.com/explore?e=eJzLLbI1VMvNzLM1UMtNrLA1NTBQS660DQhRS7Z1DQ1SKwDKpqfZliUWZaaWJOao5SfZFhRlJqeq5dsmFierlZdExwJVJFcWA-mCEgC1YxlZ&quot;, headers=headers)#, proxies=proxies)
      content = r.content
      soup = BeautifulSoup(content, &quot;html.parser&quot;)
  </code></pre>
  <p>and as I need the wine maker, wine name and ratings this is how I've tried this:</p>
  <pre><code>alls = []
      for d in soup.findAll('div', attrs={'class':'explorerCard__titleColumn--28kWX'}):
          
          Winery = d.find_all(&quot;a&quot;, attrs={&quot;class&quot;:&quot;VintageTitle_winery--2YoIr&quot;})
          Wine = d.find_all(&quot;a&quot;, attrs={&quot;class&quot;:&quot;VintageTitle_wine--U7t9G&quot;})
          Rating = d.find_all(&quot;div&quot;, attrs={&quot;class&quot;:&quot;VivinoRatingWide_averageValue--1zL_5&quot;})
          num_Reviews = d.find_all(&quot;div&quot;, attrs={&quot;class&quot;:&quot;VivinoRatingWide__basedOn--s6y0t&quot;})
          Stars = d.find_all(&quot;div&quot;, attrs={&quot;aria-label&quot;:&quot;rating__rating--ZZb_x rating__vivino--1vGCy&quot;})
  
          alll=[]
  
          if Winery is not None:
              #print(n[0][&quot;alt&quot;])
              alll.append(Winery.text)
  
          else:
              alll.append(&quot;unknown-winery&quot;)
  
          if Wine is not None:
              #print(wine.text)
              alll.append(wine.text)
          else:
              alll.append(&quot;0&quot;)
  
          if Rating is not None:
              #print(rating.text)
              alll.append(rating.text)
  
          else:
              alll.append(&quot;0&quot;)
  ...
  </code></pre>
  <p>and then getting the data into a DataFrame:</p>
  <pre><code>results = []
  for i in range(1, no_pages+1):
      results.append(get_data())
  flatten = lambda l: [item for sublist in l for item in sublist]
  df = pd.DataFrame(flatten(results),columns=['Winery','Wine','Rating','num_review', 'Stars'])
  df.to_csv('redwines.csv', index=False, encoding='utf-8')
  </code></pre>