title: Scikit Learn - Extract word tokens from a string delimiter using CountVectorizer
body: <p>I have list of strings. If any string contains the <em>'#'</em> character then I want to extract the first part of the string and get the frequency count of word tokens from this part of string only. i.e
  if the string is <em>"first question # on stackoverflow"</em>
  expected tokens are <em>"first","question"</em></p>
  
  <p>If the string does not contain <em>'#'</em> then return tokens of the whole string.</p>
  
  <p>To compute the term document matrix I am using <code>CountVectorizer</code> from scikit.</p>
  
  <p>Find below my code:</p>
  
  <pre><code>class MyTokenizer(object):
      def __call__(self,s):
          if(s.find('#')==-1):
              return s
          else:
              return s.split('#')[0]
      def FindKmeans():
          text = ["first ques # on stackoverflow", "please help"]
          vec = CountVectorizer(tokenizer=MyTokenizer(), analyzer = 'word')
          pos_vector = vec.fit_transform(text).toarray()
          print(vec.get_feature_names())`
  
  output : [u' ', u'a', u'e', u'f', u'h', u'i', u'l', u'p', u'q', u'r', u's', u't', u'u']
  
  Expected Output : [u'first', u'ques', u'please', u'help']
  </code></pre>