title: Scraping data from within multiple div classes into pandas data frame
body: <p>I am scraping some data off a dashboard and am stuck with trying to get some data within multiple <code>div classes</code> into a pandas data frame. How should I go about trying convert something like this :</p>
  
  <pre><code>[&lt;div class="map-item" data-companyname="Apical Group" data-country="INDONESIA" data-district="Jakarta Utara" data-latitude="-6.099396000" data-longitude="106.951478000" data-millname="AAJ Marunda" data-province="Jakarta" data-report="http://naturalhealthytreat.com/sites/neste-daemeter.com/files/AAJ_Marunda.pdf" id="map_item_4645"&gt;AAJ Marunda&lt;/div&gt;,
   &lt;div class="map-item" data-companyname="Apical Group" data-country="INDONESIA" data-district="Lubuk Gaung" data-latitude="1.754005000" data-longitude="101.363532000" data-millname="Sari Dumai Sejati" data-province="Riau" data-report="http://naturalhealthytreat.com/sites/neste-daemeter.com/files/Sari_Dumai_Sejati.pdf" id="map_item_4646"&gt;Sari Dumai Sejati&lt;/div&gt;,
   &lt;div class="map-item" data-companyname="Kutai Refinery Nusantara " data-country="INDONESIA" data-district="Balikpapan" data-latitude="-1.179099000" data-longitude="116.788274000" data-millname="Kutai Refinery Nusantara " data-province="Penajam Paser Utara" data-report="http://naturalhealthytreat.com/sites/neste-daemeter.com/files/Kutai_Refinery_Nusantara_.pdf" id="map_item_4647"&gt;Kutai Refinery Nusantara &lt;/div&gt;]
  </code></pre>
  
  <p>Into a data frame like this:</p>
  
  <pre><code>no  companyname country district    latitude    longitude   millname    province    report
  1   Apical Group    INDONESIA   Jakarta Utara   -6.099396   106.951478  AAJ Marunda Jakarta http://naturalhealthytreat.com/sites/neste-daemeter.com/files/AAJ_Marunda.pdf
  2   Apical Group    INDONESIA   Lubuk Gaung 1.754005    101.363532  Sari Dumai Sejati   Riau    http://naturalhealthytreat.com/sites/neste-daemeter.com/files/Kutai_Refinery_Nusantara_.pdf
  3   Kutai Refinery Nusantara    INDONESIA   Balikpapan  -1.179099   116.788274  Kutai Refinery Nusantara    Penajam Paser Utara http://naturalhealthytreat.com/sites/neste-daemeter.com/files/Kutai_Refinery_Nusantara_.pdf
  </code></pre>
  
  <p>So far, this is code I have put together to get the multiple div classes from the web page:</p>
  
  <pre><code>from bs4 import BeautifulSoup
  import requests
  
  # Link of Neste dashboard
  url = 'http://nestetraceabilitydashboard.com/nestes-pfad-traceability-dashboard' 
  page = requests.get(url).content
  soup = BeautifulSoup(page, "html.parser")
  
  divList = soup.findAll('div', attrs={ "class" : "map-item"})
  </code></pre>