title: Retrieve TF-IDF Values for Textual Documents in CSV File
body: <p>I have a CSV file with two columns (no header), held in a variable known as 'dataset': </p>
  
  <pre><code>         Year      Document Text
    0     ['1991']  ['FACTSHEET ', 'WHAT ', 'IS ', 'AIDS', 'AIDS '...
    1     ['1991']  ['HIV ', 'IT', "'S ", 'YOUR ', 'CHOICE', 'Ever...
    2     ['1991']  ['ACET ', 'AIDS ', 'CARE ', 'EDUCATION ', 'AND...
  </code></pre>
  
  <p>I'm attempting to construct a Bag of Words model using Scikit-learn and gather the weightings using TF-IDF. However, I'm having difficulty with obtaining actual results, as the output of the code below returns 2480 rows (correct) * 346862 columns (corrected by @Jarad). I would appreciate someone helping me decipher these results, and point me in the right direction as to their formatting (to provide clarity) or correction (to provide validity) so that I can progress towards the later stages of a Bag of Words model implementation.</p>
  
  <p>Python Code:</p>
  
  <pre><code>from sklearn.feature_extraction.text import TfidfVectorizer
  v = TfidfVectorizer()
  x = v.fit_transform(dataset.iloc[:,1])
  
  df1 = pd.DataFrame(x.toarray(), columns=v.get_feature_names())
  print(df1)
  </code></pre>
  
  <p>Output:</p>
  
  <pre><code>            00       000      0000  00000  00000000  00000001  0000001  00001  
  0     0.000000  0.011453  0.000000    0.0       0.0       0.0      0.0    0.0   
  1     0.000000  0.022032  0.000000    0.0       0.0       0.0      0.0    0.0   
  2     0.006352  0.009717  0.000000    0.0       0.0       0.0      0.0    0.0   
  3     0.001422  0.015949  0.000000    0.0       0.0       0.0      0.0    0.0   
  4     0.000000  0.002377  0.000000    0.0       0.0       0.0      0.0    0.0 
  </code></pre>
  
  <p>Should I tokenize the document prior to storing it in the CSV file? I decided against it due to the fact that I would hope to analyse sentence structure at a later stage as well.</p>