title: I have issues with tokenizing a list of files that I read from a directory. Below is my code
body: <pre><code>import glob
  import os
  import nltk.data
  from nltk.sentiment.vader import SentimentIntensityAnalyzer
  from nltk import sentiment
  from nltk import word_tokenize
  
  sid = SentimentIntensityAnalyzer()
  
  
  tokenizer = nltk.data.load('tokenizers/punkt/english.pickle')
  
  
  files = glob.glob(os.path.join(os.getcwd(), 'speeches', '*.txt'))
  
  
  text = []
  
  for file in files:
      #open the file and then call .read() to get the text
  
  with open(file) as f:
          text.append(f.read())
  
  sentences = tokenizer.tokenize(text)
  </code></pre>
  
  <p>and printing polarity scores for each one.</p>
  
  <pre><code>for sentence in sentences:
          print(sentence,  file=open('sentiment1.txt', 'a'))
          scores = sid.polarity_scores(sentence)
          for key in sorted(scores):
                  print('{0}: {1}, '.format(key, scores[key]), end='', file=open('sentiment1.txt', 'a'))
          print()
  </code></pre>
  
  <p>I have five text files in the 'speeches' folder which was successfully read but the issue is that the output file (text) is of list type instead of string and tokenize expects a string or byte as its argument. Please assist me with a tweak as I have read that this might be an issue with Python 3. Thanks</p>