title: For Tokensregex, does a rule need to be token type to use Annotate?
body: <p>I am going over some older code Tokensregex code and I am faced with a situation where some characters are not being tokenized by the PTBTokenizer.  In particular, I am looking at currency symbols.  So for example, ₱ would not be a token whereas some others are such as $ would.</p>
  
  <p>Well I want to try to write a text type rule instead of token type to try to catch this symbol otherwise in a capture group and then do something like <code>Annotate($0, ner, "MONEY")</code> to capture a string such as ₱240.</p>
  
  <p>When I attempt this I get:</p>
  
  <blockquote>
    <p>... 49 more Caused by: java.lang.ClassCastException:
    edu.stanford.nlp.ling.tokensregex.TokenSequencePattern cannot be cast
    to java.lang.String   at
    edu.stanford.nlp.ling.tokensregex.SequenceMatchRules$TextPatternExtractRuleCreator.create(SequenceMatchRules.java:666)
      at
    edu.stanford.nlp.ling.tokensregex.SequenceMatchRules.createExtractionRule(SequenceMatchRules.java:331)
      at
    edu.stanford.nlp.ling.tokensregex.SequenceMatchRules.createRule(SequenceMatchRules.java:321)
      at
    edu.stanford.nlp.ling.tokensregex.parser.TokenSequenceParser.Rule(TokenSequenceParser.java:141)
      at
    edu.stanford.nlp.ling.tokensregex.parser.TokenSequenceParser.RuleList(TokenSequenceParser.java:125)
      at
    edu.stanford.nlp.ling.tokensregex.parser.TokenSequenceParser.updateExpressionExtractor(TokenSequenceParser.java:32)
      at
    edu.stanford.nlp.ling.tokensregex.CoreMapExpressionExtractor.createExtractorFromFiles(CoreMapExpressionExtractor.java:292)
      ... 52 more</p>
  </blockquote>
  
  <p>So can I do the above, create a MONEY ner annotation. if the currency symbol is missed by the tokenizer?</p>
  
  <p><strong>EXAMPLE</strong></p>
  
  <p>Text rule attempt at doing what I want (create ner annotation called CURRENCY for a string containing a peso monetary value)</p>
  
  <pre><code>ENV.defaults["ruleType"] = "text"
  { text: /(₱\d+)/ =&gt; Annotate($0, ner, "CURRENCY")}
  </code></pre>
  
  <p>Token rule successfully doing what I want (because yen is a recoginized token).  This creates a yen monetary string with ner annotation of CURRENCY.</p>
  
  <pre><code>ENV.defaults["ruleType"] = "tokens"
  ENV.defaults["matchWithResults"] = TRUE
  
  # Set default string pattern flags (to case-insensitive)
  ENV.defaultStringPatternFlags = 2
  
  ENV.defaults["stage"] = 0
  
  # Ex: ¥3000
  {   
  pattern:  ([{ word: "¥" }] $NUMBER_COMMA_SEP $LARGE_NUMBERS?),
  action: (Annotate($0, ner, "CURRENCY"))
  }
  </code></pre>
  
  <p>ner is defined as:</p>
  
  <pre><code>ner = { type: "CLASS", value: "edu.stanford.nlp.ling.CoreAnnotations$NamedEntityTagAnnotation" }
  </code></pre>
  
  <p>and then:</p>
  
  <pre><code>$NUMBER_COMMA_SEP = "$NUMBER_NON_CD | ([{ tag: /CD/ } &amp; $VALID_NUMERIC_CHARS] [{ tag: /CD/; word: /,\d+(\.\d+)?/ }]*)"
  $LARGE_NUMBERS = "/thousand|million|mil|mn|billion|bil|bn|trillion/"
  </code></pre>