class_docstr: TextExplainer allows to explain predictions of black-box text classifiers
  using LIME algorithm.
  
  Parameters
  ----------
  n_samples : int
      A number of samples to generate and train on. Default is 5000.
  
      With larger n_samples it takes more CPU time and RAM to explain
      a prediction, but it could give better results. Larger n_samples
      could be also required to get good results if you don't want to
      make strong assumptions about the black-box classifier
      (e.g. char_based=True and position_dependent=True).
  char_based : bool
      True if explanation should be char-based, False if it should be
      token-based. Default is False.
  clf : object, optional
      White-box probabilistic classifier. It should be supported by eli5,
      follow scikit-learn interface and provide predict_proba method.
      When not set, a default classifier is used (logistic regression with
      elasticnet regularization trained with SGD).
  vec : object, optional
      Vectorizer which converts generated texts to feature vectors
      for the white-box classifier. When not set, a default vectorizer is
      used; which one depends on ``char_based`` and ``position_dependent``
      arguments.
  sampler : MaskingTextSampler or MaskingTextSamplers, optional
      Sampler used to generate modified versions of the text.
  position_dependent : bool
      When True, a special vectorizer is used which takes
      each token or character (depending on ``char_based`` value)
      in account separately. When False (default) a vectorized passed in
      ``vec`` or a default vectorizer is used.
  
      Default vectorizer converts text to vector using bag-of-ngrams
      or bag-of-char-ngrams approach (depending on ``char_based`` argument).
      It means that it may be not powerful enough to approximate a black-box
      classifier which e.g. takes in account word FOO in the beginning of
      the document, but not in the end.
  
      When ``position_dependent`` is True the model becomes powerful enough
      to account for that, but it can become more noisy and require
      larger ``n_samples`` to get an OK explanation.
  
      When ``char_based=False`` the default vectorizer uses word bigrams
      in addition to unigrams; this is less powerful than
      ``position_dependent=True``, but can give similar results in practice.
  rbf_sigma : float, optional
      Sigma parameter of RBF kernel used to post-process cosine similarity
      values. Default is None, meaning no post-processing
      (cosine simiilarity is used as sample weight as-is).
      Small ``rbf_sigma`` values (e.g. 0.1) tell the classifier to pay
      more attention to generated texts which are close to the original text.
      Large ``rbf_sigma`` values (e.g. 1.0) make distance between text
      irrelevant.
  
      Note that if you're using large ``rbf_sigma`` it could be more
      efficient to use custom ``samplers`` instead, in order to generate
      text samples which are closer to the original text in the first place.
      Use e.g. ``max_replace`` parameter of :class:`~.MaskingTextSampler`.
  random_state : integer or numpy.random.RandomState, optional
      random state
  expand_factor : int or None
      To approximate output of the probabilistic classifier generated
      dataset is expanded by ``expand_factor`` (10 by default)
      according to the predicted label probabilities. This is a workaround
      for scikit-learn limitation (no cross-entropy loss for non 1/0 labels).
      With larger values training takes longer, but probability output
      can be approximated better.
  
      expand_factor=None turns this feature off; pass None when you know
      that black-box classifier returns only 1.0 or 0.0 probabilities.
  token_pattern : str, optional
      Regex which matches a token. Use it to customize tokenization.
      Default value depends on ``char_based`` parameter.
  
  Attributes
  ----------
  rng_ : numpy.random.RandomState
      random state
  
  samples_ : list[str]
      A list of samples the local model is trained on.
      Only available after :func:`fit`.
  
  X_ : ndarray or scipy.sparse matrix
      A matrix with vectorized ``samples_``.
      Only available after :func:`fit`.
  
  similarity_ : ndarray
      Similarity vector. Only available after :func:`fit`.
  
  y_proba_ : ndarray
      probabilities predicted by black-box classifier
      (``predict_proba(self.samples_)`` result).
      Only available after :func:`fit`.
  
  clf_ : object
      Trained white-box classifier. Only available after :func:`fit`.
  
  vec_ : object
      Fit white-box vectorizer. Only available after :func:`fit`.
  
  metrics_ : dict
      A dictionary with metrics of how well the local
      classification pipeline approximates the black-box pipeline.
      Only available after :func:`fit`.
class_docstr_tok: ["TextExplainer","allows","to","explain","predictions","of","black-box","text","classifiers","using","LIME","algorithm",".","Parameters","--","--","--","--","--","n_samples",":","int","A","number","of","samples","to","generate","and","train","on",".","Default","is","5000",".","With","larger","n_samples","it","takes","more","CPU","time","and","RAM","to","explain","a","prediction",",","but","it","could","give","better","results",".","Larger","n_samples","could","be","also","required","to","get","good","results","if","you","do","n't","want","to","make","strong","assumptions","about","the","black-box","classifier","(","e.g",".","char_based=True","and","position_dependent=True",")",".","char_based",":","bool","True","if","explanation","should","be","char-based",",","False","if","it","should","be","token-based",".","Default","is","False",".","clf",":","object",",","optional","White-box","probabilistic","classifier",".","It","should","be","supported","by","eli5",",","follow","scikit-learn","interface","and","provide","predict_proba","method",".","When","not","set",",","a","default","classifier","is","used","(","logistic","regression","with","elasticnet","regularization","trained","with","SGD",")",".","vec",":","object",",","optional","Vectorizer","which","converts","generated","texts","to","feature","vectors","for","the","white-box","classifier",".","When","not","set",",","a","default","vectorizer","is","used",";","which","one","depends","on","``","char_based","``","and","``","position_dependent","``","arguments",".","sampler",":","MaskingTextSampler","or","MaskingTextSamplers",",","optional","Sampler","used","to","generate","modified","versions","of","the","text",".","position_dependent",":","bool","When","True",",","a","special","vectorizer","is","used","which","takes","each","token","or","character","(","depending","on","``","char_based","``","value",")","in","account","separately",".","When","False","(","default",")","a","vectorized","passed","in","``","vec","``","or","a","default","vectorizer","is","used",".","Default","vectorizer","converts","text","to","vector","using","bag-of-ngrams","or","bag-of-char-ngrams","approach","(","depending","on","``","char_based","``","argument",")",".","It","means","that","it","may","be","not","powerful","enough","to","approximate","a","black-box","classifier","which","e.g",".","takes","in","account","word","FOO","in","the","beginning","of","the","document",",","but","not","in","the","end",".","When","``","position_dependent","``","is","True","the","model","becomes","powerful","enough","to","account","for","that",",","but","it","can","become","more","noisy","and","require","larger","``","n_samples","``","to","get","an","OK","explanation",".","When","``","char_based=False","``","the","default","vectorizer","uses","word","bigrams","in","addition","to","unigrams",";","this","is","less","powerful","than","``","position_dependent=True","``",",","but","can","give","similar","results","in","practice",".","rbf_sigma",":","float",",","optional","Sigma","parameter","of","RBF","kernel","used","to","post-process","cosine","similarity","values",".","Default","is","None",",","meaning","no","post-processing","(","cosine","simiilarity","is","used","as","sample","weight","as-is",")",".","Small","``","rbf_sigma","``","values","(","e.g",".","0.1",")","tell","the","classifier","to","pay","more","attention","to","generated","texts","which","are","close","to","the","original","text",".","Large","``","rbf_sigma","``","values","(","e.g",".","1.0",")","make","distance","between","text","irrelevant",".","Note","that","if","you","'re","using","large","``","rbf_sigma","``","it","could","be","more","efficient","to","use","custom","``","samplers","``","instead",",","in","order","to","generate","text","samples","which","are","closer","to","the","original","text","in","the","first","place",".","Use","e.g",".","``","max_replace","``","parameter","of",":","class",":","`","~.MaskingTextSampler","`",".","random_state",":","integer","or","numpy.random.RandomState",",","optional","random","state","expand_factor",":","int","or","None","To","approximate","output","of","the","probabilistic","classifier","generated","dataset","is","expanded","by","``","expand_factor","``","(","10","by","default",")","according","to","the","predicted","label","probabilities",".","This","is","a","workaround","for","scikit-learn","limitation","(","no","cross-entropy","loss","for","non","1\/0","labels",")",".","With","larger","values","training","takes","longer",",","but","probability","output","can","be","approximated","better",".","expand_factor=None","turns","this","feature","off",";","pass","None","when","you","know","that","black-box","classifier","returns","only","1.0","or","0.0","probabilities",".","token_pattern",":","str",",","optional","Regex","which","matches","a","token",".","Use","it","to","customize","tokenization",".","Default","value","depends","on","``","char_based","``","parameter",".","Attributes","--","--","--","--","--","rng_",":","numpy.random.RandomState","random","state","samples_",":","list","[","str","]","A","list","of","samples","the","local","model","is","trained","on",".","Only","available","after",":","func",":","`","fit","`",".","X_",":","ndarray","or","scipy.sparse","matrix","A","matrix","with","vectorized","``","samples_","``",".","Only","available","after",":","func",":","`","fit","`",".","similarity_",":","ndarray","Similarity","vector",".","Only","available","after",":","func",":","`","fit","`",".","y_proba_",":","ndarray","probabilities","predicted","by","black-box","classifier","(","``","predict_proba","(","self.samples_",")","``","result",")",".","Only","available","after",":","func",":","`","fit","`",".","clf_",":","object","Trained","white-box","classifier",".","Only","available","after",":","func",":","`","fit","`",".","vec_",":","object","Fit","white-box","vectorizer",".","Only","available","after",":","func",":","`","fit","`",".","metrics_",":","dict","A","dictionary","with","metrics","of","how","well","the","local","classification","pipeline","approximates","the","black-box","pipeline",".","Only","available","after",":","func",":","`","fit","`","."]
code: def __init__(
      self,
      n_samples=5000,  # type: int
      char_based=None,  # type: bool
      clf=None,
      vec=None,
      sampler=None,  # type: BaseSampler
      position_dependent=False,  # type: bool
      rbf_sigma=None,  # type: float
      random_state=None,
      expand_factor=10,  # type: Optional[int]
      token_pattern=None,  # type: Optional[str]
  ):
      # type: (...) -> None
      self.n_samples = n_samples
      self.random_state = random_state
      self.expand_factor = expand_factor
      self.rng_ = check_random_state(random_state)
      if clf is None:
          clf = self._default_clf()
      self.clf = clf
  
      if char_based is None:
          if token_pattern is None:
              self.char_based = False  # type: Optional[bool]
              self.token_pattern = DEFAULT_TOKEN_PATTERN  # type: str
          else:
              self.char_based = None
              self.token_pattern = token_pattern
      else:
          if token_pattern is None:
              token_pattern = (
                  CHAR_TOKEN_PATTERN
                  if char_based
                  else DEFAULT_TOKEN_PATTERN
              )
          self.char_based = char_based
          self.token_pattern = token_pattern
  
      if sampler is None:
          sampler = MaskingTextSamplers(
              sampler_params=[{"bow": False}, {"bow": True}],
              weights=[0.7, 0.3],
              token_pattern=self.token_pattern,
              random_state=self.rng_,
          )
      self.sampler = sampler
      self.rbf_sigma = rbf_sigma
      self.position_dependent = position_dependent
      if position_dependent:
          if vec is not None:
              raise ValueError(
                  "Custom vectorizers are only supported with "
                  "position_dependent=False (because "
                  "position_dependent=True uses its own "
                  "vectorizer)"
              )
      else:
          if vec is None:
              if self.char_based:
                  vec = CountVectorizer(
                      analyzer="char",
                      ngram_range=(2, 5),
                  )
              else:
                  vec = CountVectorizer(
                      token_pattern=self.token_pattern,
                      ngram_range=(1, 2),
                  )
          self.vec = vec
  
imports: from __future__ import absolute_import
  from typing import Any, Callable, Dict, Optional
  import numpy
  from sklearn.feature_extraction.text import CountVectorizer
  from sklearn.linear_model import SGDClassifier
  from sklearn.model_selection import train_test_split
  from sklearn.utils import check_random_state
  from sklearn.base import clone, BaseEstimator
  import eli5
  from eli5.sklearn.utils import sklearn_version
  from eli5.lime.samplers import BaseSampler
  from eli5.lime.textutils import DEFAULT_TOKEN_PATTERN, CHAR_TOKEN_PATTERN
  from eli5.lime.samplers import MaskingTextSamplers
  from eli5.lime.utils import fit_proba, score_with_sample_weight, mean_kl_divergence, fix_multiclass_predict_proba, rbf
  from eli5.lime._vectorizer import SingleDocumentVectorizer
label_desc: Use image node_id 1 to create a new TextExplainer object from inherited base classes: BaseEstimator with example: obj = TextExplainer(n_samples, char_based, clf, vec, sampler, position_dependent, rbf_sigma, random_state, expand_factor, token_pattern)