title: Scala stop words
body: <p>I would like to remove the following occurences from my tweet data:</p>
  
  <p><strong>anything with an @ (e.g. @nike)</strong></p>
  
  <p><strong>anything starting with ://</strong></p>
  
  <p>Within my scala script I have stopwords, but they must match exactly the output. Is there a way to add a stopword such as @* or ://* which would account for all the possibilities of words I want to remove?</p>
  
  <pre><code>val source = CSVFile("output.csv")
  
  val tokenizer = {
  SimpleEnglishTokenizer() ~&gt;            // tokenize on space and punctuation
  WordsAndNumbersOnlyFilter() ~&gt;         // ignore non-words and non-numbers
  CaseFolder() ~&gt;                        // lowercase everything
  MinimumLengthFilter(3)                 // take terms with &gt;=3 characters 
  }
  
  val text = {
  source ~&gt;                              // read from the source file
  Column(1) ~&gt;                           // select column containing text
  TokenizeWith(tokenizer) ~&gt;             // tokenize with tokenizer above
  TermCounter() ~&gt;                       // collect counts (needed below)
  TermMinimumDocumentCountFilter(30) ~&gt;   // filter terms in &lt;4 docs
  TermStopListFilter(List("a", "and", "I", "but", "what")) ~&gt; // stopword list
  TermDynamicStopListFilter(10) ~&gt;       // filter out 30 most common terms  
  DocumentMinimumLengthFilter(5)         // take only docs with &gt;=5 terms 
  }
  </code></pre>
  
  <p>The Tokenizer doesn't seem to pick up on these non-letter characters. However it filters out # without problem.
  Thanks for your help!</p>