class_docstr: Uses a regular expression to extract tokens from text.
  
  >>> rex = RegexTokenizer()
  >>> [token.text for token in rex(u("hi there 3.141 big-time under_score"))]
  ["hi", "there", "3.141", "big", "time", "under_score"]
docstr: :param expression: A regular expression object or string. Each match
      of the expression equals a token. Group 0 (the entire matched text)
      is used as the text of the token. If you require more complicated
      handling of the expression match, simply write your own tokenizer.
  :param gaps: If True, the tokenizer *splits* on the expression, rather
      than matching on the expression.
docstr_tok: [":","param","expression",":","A","regular","expression","object","or","string",".","Each","match","of","the","expression","equals","a","token",".","Group","0","(","the","entire","matched","text",")","is","used","as","the","text","of","the","token",".","If","you","require","more","complicated","handling","of","the","expression","match",",","simply","write","your","own","tokenizer",".",":","param","gaps",":","If","True",",","the","tokenizer","*","splits","*","on","the","expression",",","rather","than","matching","on","the","expression","."]
code: def __init__(self, expression=default_pattern, gaps=False):
      """
      :param expression: A regular expression object or string. Each match
          of the expression equals a token. Group 0 (the entire matched text)
          is used as the text of the token. If you require more complicated
          handling of the expression match, simply write your own tokenizer.
      :param gaps: If True, the tokenizer *splits* on the expression, rather
          than matching on the expression.
      """
  
      self.expression = rcompile(expression)
      self.gaps = gaps
  
imports: from whoosh.compat import u, text_type
  from whoosh.analysis.acore import Composable, Token
  from whoosh.util.text import rcompile
label_desc: Use image node_id 1 to create a new RegexTokenizer object from inherited base classes: Tokenizer with example: obj = RegexTokenizer(expression, gaps)