title: How to use stopword in the preprocessing a txt file
body: <p>I'm  trying to write a code that will text process and eventually index all of them. I first need to remove non-alphabetic characters and punctuation, and convert capital letter to lower case, then remove the stopwords.</p>
  
  <p>Here is what I did so far:</p>
  
  <pre><code>from stopwords import *
  
  def removeStopwords(wordlist, flag):
      return [w for w in wordlist if w not in flag]
  
  def preprocessing():
      import re
      with open('44.txt', 'r', encoding = 'utf8') as data:
          for line in data:
              a = line.rstrip().lower()
              result = re.sub('[^a-zA-Z]', ' ', a)
              b = removeStopwords(result, stopwords)
              print(b)
  
  if __name__ == '__main__':
      preprocessing()
  </code></pre>
  
  <p>Then I get all the letters break into parts like
   ['a'], ['w'], ['o'], ['l'], ['f'] </p>
  
  <p>stopwords.py is just list of words like:</p>
  
  <pre><code>stopwords = ['a', 'are', 'aren t', ....]
  </code></pre>
  
  <p>Can somebody tell me what is going on?</p>
  
  <p>Thanks for your time !</p>