title: Python - Web Scraping HTML table and printing to CSV
body: <p>I'm pretty much brand new to Python, but I'm looking to build a webscraping tool that will rip data from an HTML table online and print it into a CSV in the same format.</p>
  
  <p>Here's a sample of the HTML table (it's enormous, so I'm going to provide only a few rows).</p>
  
  <pre><code>&lt;div class="col-xs-12 tab-content"&gt;
          &lt;div id="historical-data" class="tab-pane active"&gt;
            &lt;div class="tab-header"&gt;
              &lt;h2 class="pull-left bottom-margin-2x"&gt;Historical data for Bitcoin&lt;/h2&gt;
  
              &lt;div class="clear"&gt;&lt;/div&gt;
  
              &lt;div class="row"&gt;
                &lt;div class="col-md-12"&gt;
                  &lt;div class="pull-left"&gt;
                    &lt;small&gt;Currency in USD&lt;/small&gt;
                  &lt;/div&gt;
                  &lt;div id="reportrange" class="pull-right"&gt;
                      &lt;i class="glyphicon glyphicon-calendar fa fa-calendar"&gt;&lt;/i&gt;&amp;nbsp;
                      &lt;span&gt;Aug 16, 2017 - Sep 15, 2017&lt;/span&gt; &lt;b class="caret"&gt;&lt;/b&gt;
                  &lt;/div&gt;
                &lt;/div&gt;
              &lt;/div&gt;
  
              &lt;table class="table"&gt;
                &lt;thead&gt;
                &lt;tr&gt;
                  &lt;th class="text-left"&gt;Date&lt;/th&gt;
                  &lt;th class="text-right"&gt;Open&lt;/th&gt;
                  &lt;th class="text-right"&gt;High&lt;/th&gt;
                  &lt;th class="text-right"&gt;Low&lt;/th&gt;
                  &lt;th class="text-right"&gt;Close&lt;/th&gt;
                  &lt;th class="text-right"&gt;Volume&lt;/th&gt;
                  &lt;th class="text-right"&gt;Market Cap&lt;/th&gt;
                &lt;/tr&gt;
                &lt;/thead&gt;
                &lt;tbody&gt;
  
                  &lt;tr class="text-right"&gt;
                    &lt;td class="text-left"&gt;Sep 14, 2017&lt;/td&gt;
                    &lt;td&gt;3875.37&lt;/td&gt;     
                    &lt;td&gt;3920.60&lt;/td&gt;
                    &lt;td&gt;3153.86&lt;/td&gt;
                    &lt;td&gt;3154.95&lt;/td&gt;
                    &lt;td&gt;2,716,310,000&lt;/td&gt;
                    &lt;td&gt;64,191,600,000&lt;/td&gt;
                  &lt;/tr&gt;
  
                  &lt;tr class="text-right"&gt;
                    &lt;td class="text-left"&gt;Sep 13, 2017&lt;/td&gt;
                    &lt;td&gt;4131.98&lt;/td&gt;     
                    &lt;td&gt;4131.98&lt;/td&gt;
                    &lt;td&gt;3789.92&lt;/td&gt;
                    &lt;td&gt;3882.59&lt;/td&gt;
                    &lt;td&gt;2,219,410,000&lt;/td&gt;
                    &lt;td&gt;68,432,200,000&lt;/td&gt;
                  &lt;/tr&gt;
  
                  &lt;tr class="text-right"&gt;
                    &lt;td class="text-left"&gt;Sep 12, 2017&lt;/td&gt;
                    &lt;td&gt;4168.88&lt;/td&gt;     
                    &lt;td&gt;4344.65&lt;/td&gt;
                    &lt;td&gt;4085.22&lt;/td&gt;
                    &lt;td&gt;4130.81&lt;/td&gt;
                    &lt;td&gt;1,864,530,000&lt;/td&gt;
                    &lt;td&gt;69,033,400,000&lt;/td&gt;
                  &lt;/tr&gt;                
                &lt;/tbody&gt;
              &lt;/table&gt;
            &lt;/div&gt;
  
          &lt;/div&gt;
      &lt;/div&gt;
  </code></pre>
  
  <p>I'm particularly interested in recreating the table with the same column headers provided: "Date," "Open," "High," "Low," "Close," "Volume," "Market Cap." Currently, I've been able to write a simple script that will essentially go to the URL, download the HTML, parse with BeautifulSoup, and then use 'for' statements to get the td elements. Below a sample of my code (URL omitted) and the result:</p>
  
  <pre><code>from bs4 import BeautifulSoup
  import requests
  import pandas as pd
  import csv
  
  url = "enterURLhere"
  page = requests.get(url)
  pagetext = page.text
  
  pricetable = {
      "Date" : [],
      "Open" : [],
      "High" : [],
      "Low" : [],
      "Close" : [],
      "Volume" : [],
      "Market Cap" : []
  }
  
  soup = BeautifulSoup(pagetext, 'html.parser')
  
  file = open("test.csv", 'w')
  
  for row in soup.find_all('tr'):
      for col in row.find_all('td'):
          print(col.text)
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/sSKHj.jpg" rel="nofollow noreferrer">sample output</a></p>
  
  <p>Anyone have any pointers on how to at least reformat the data pull into the table? Thanks.</p>