class_docstr: Model selection across several estimators and preprocessing pipelines.
  
  The :class:`Evaluator` allows users to evaluate several models in one call
  across a set preprocessing pipelines. The class is useful for comparing
  a set of estimators, especially when several preprocessing pipelines is to
  be evaluated. By pre-making all folds and iteratively fitting estimators
  with different parameter settings, array slicing and preprocessing is kept
  to a minimum. This can greatly reduced fit time compared to
  creating pipeline classes for each estimator and pipeline and fitting them
  one at a time in an Scikit-learn
  :class:`sklearn.model_selection.GridSearch` class.
  
  Preprocessing can be done before making any evaluation, and several
  evaluations can be made on the pre-made folds. Current implementation
  relies on a randomized grid search, so parameter grids must be specified as
  SciPy distributions (or a class that accepts an ``rvs`` method).
  
  .. versionchanged:: 0.2.0
  
  Parameters
  ----------
  scorer : function
      a scoring function that follows the Scikit-learn API::
  
          score = scorer(estimator, y_true, y_pred)
  
      A user defines scoring function, ``score = f(y_true, y_pred)`` can be
      made into a scorer by calling on the ML-Ensemble implementation of
      Scikit-learn's ``make_scorer``. NOTE: do **not** use Scikit-learn's
      ``make_scorer`` if the Evaluator is to be pickled. ::
  
          from mlens.metrics import make_scorer
          scorer = make_scorer(scoring_function, **kwargs)
  
  error_score : int, optional
      score to assign when fitting an estimator fails. If ``None``, the
      evaluator will raise an error.
  
  cv : int or obj, default = 2
      cross validation folds to use. Either pass a ``KFold`` class
      that obeys the Scikit-learn API.
  
  metrics : list, optional
      list of aggregation metrics to calculate on scores. Default is
      mean and standard deviation.
  
  shuffle : bool, default = True
      whether to shuffle input data before creating cv folds.
  
  random_state : int, optional
      seed for creating folds (if shuffled) and parameter draws
  
  n_jobs: int, default = -1
      number of CPU cores to use.
  
  verbose : bool or int, default = False
      level of printed messages. Levels:
  
          #. ``verbose=1``: Message at start and end with total time
          #. ``verbose=2``: Additional messages for each sub-job \
             (preprocess and evaluation)
          #. ``verbose in [3, 14]``: Additional messages with job \
             completion status at increasing increasing frequency
          #. ``Verbose >= 15``: prints each job completed as \
             [case].[est].[draw].[fold]
  
      If ``verbose>=20``, prints to ``sys.stderr``, else ``sys.stdout``.
docstr: Fit
  
  Fit preprocessing if applicable and evaluate estimators if applicable.
  The method automatically determines whether to only run preprocessing,
  only evaluation (possibly on previously fitted preprocessing), or both.
  Calling ``fit`` will overwrite previously stored data where applicable.
  
  Parameters
  ----------
  X : array-like, shape=[n_samples, n_features]
      input data to preprocess and create folds from.
  
  y : array-like, shape=[n_samples, ]
      training labels.
  
  estimators : list or dict, optional
      set of estimators to use. If no preprocessing is desired or if
      only on preprocessing pipeline should apply to all, pass a list of
      estimators. The list can contain elements of named tuples
      (i.e. ``('my_name', my_est)``).
  
      If different estimators should be mapped to preprocessing cases,
      a dictionary that maps estimators to each case should
      be passed: ``{'case_a': list_of_est, ...}``.
  
  param_dicts : dict, optional
      parameter distribution mapping for estimators. Current
      implementation only supports randomized grid search. Passed
      distribution object must have an ``rvs`` method.
      See :mod:`scipy.stats` for details.
  
      There is quite some flexibility in specifying ``param_dicts``. If
      there is no preprocessing, or if all estimators are fitted on all
      preprocessing cases, the ``param_dict`` should have keys matching
      the names of the estimators. ::
  
          estimators = [('name', est), est]
  
          param_dicts = {'name': {'param-1': some_distribution},
                         'est': {'param-1': some_distribution}
                        }
  
      It is possible to specify different distributions for some or all
      preprocessing cases::
  
          preprocessing = {'case-1': transformer_list,
                           'case-2': transformer_list}
  
          estimators = [('name', est), est]
  
          param_dicts = {'name':
                             {'param-1': some_distribution},
                         ('case-1', 'est'):
                             {'param-1': some_distribution}
                         ('case-2', 'est'):
                             {'param-1': some_distribution,
                              'param-2': some_distribution}
                        }
  
      If estimators are mapped on a per-preprocessing case basis as a
      dictionary, ``param_dict`` must have key entries of the form
      ``(case_name, est_name)``.
  
  n_iter : int
      number of parameter draws to evaluate.
  
  preprocessing : dict, optional
      preprocessing cases to consider. Pass a dictionary mapping a
      case name to a preprocessing pipeline. ::
  
          preprocessing = {'case_name': transformer_list,}
  
  Returns
  -------
  self : instance
      class instance with stored estimator evaluation results in
      the ``results`` attribute.
docstr_tok: ["Fit","Fit","preprocessing","if","applicable","and","evaluate","estimators","if","applicable",".","The","method","automatically","determines","whether","to","only","run","preprocessing",",","only","evaluation","(","possibly","on","previously","fitted","preprocessing",")",",","or","both",".","Calling","``","fit","``","will","overwrite","previously","stored","data","where","applicable",".","Parameters","--","--","--","--","--","X",":","array-like",",","shape=","[","n_samples",",","n_features","]","input","data","to","preprocess","and","create","folds","from",".","y",":","array-like",",","shape=","[","n_samples",",","]","training","labels",".","estimators",":","list","or","dict",",","optional","set","of","estimators","to","use",".","If","no","preprocessing","is","desired","or","if","only","on","preprocessing","pipeline","should","apply","to","all",",","pass","a","list","of","estimators",".","The","list","can","contain","elements","of","named","tuples","(","i.e",".","``","(","'my_name","'",",","my_est",")","``",")",".","If","different","estimators","should","be","mapped","to","preprocessing","cases",",","a","dictionary","that","maps","estimators","to","each","case","should","be","passed",":","``","{","'case_a","'",":","list_of_est",",","...","}","``",".","param_dicts",":","dict",",","optional","parameter","distribution","mapping","for","estimators",".","Current","implementation","only","supports","randomized","grid","search",".","Passed","distribution","object","must","have","an","``","rvs","``","method",".","See",":","mod",":","`","scipy.stats","`","for","details",".","There","is","quite","some","flexibility","in","specifying","``","param_dicts","``",".","If","there","is","no","preprocessing",",","or","if","all","estimators","are","fitted","on","all","preprocessing","cases",",","the","``","param_dict","``","should","have","keys","matching","the","names","of","the","estimators",".",":",":","estimators","=","[","(","'name","'",",","est",")",",","est","]","param_dicts","=","{","'name","'",":","{","'param-1","'",":","some_distribution","}",",","'est","'",":","{","'param-1","'",":","some_distribution","}","}","It","is","possible","to","specify","different","distributions","for","some","or","all","preprocessing","cases",":",":","preprocessing","=","{","'case-1","'",":","transformer_list",",","'case-2","'",":","transformer_list","}","estimators","=","[","(","'name","'",",","est",")",",","est","]","param_dicts","=","{","'name","'",":","{","'param-1","'",":","some_distribution","}",",","(","'case-1","'",",","'est","'",")",":","{","'param-1","'",":","some_distribution","}","(","'case-2","'",",","'est","'",")",":","{","'param-1","'",":","some_distribution",",","'param-2","'",":","some_distribution","}","}","If","estimators","are","mapped","on","a","per-preprocessing","case","basis","as","a","dictionary",",","``","param_dict","``","must","have","key","entries","of","the","form","``","(","case_name",",","est_name",")","``",".","n_iter",":","int","number","of","parameter","draws","to","evaluate",".","preprocessing",":","dict",",","optional","preprocessing","cases","to","consider",".","Pass","a","dictionary","mapping","a","case","name","to","a","preprocessing","pipeline",".",":",":","preprocessing","=","{","'case_name","'",":","transformer_list",",","}","Returns","--","--","--","-","self",":","instance","class","instance","with","stored","estimator","evaluation","results","in","the","``","results","``","attribute","."]
code: def fit(
      self,
      X,
      y,
      estimators=None,
      param_dicts=None,
      n_iter=2,
      preprocessing=None,
  ):
      """Fit
  
      Fit preprocessing if applicable and evaluate estimators if applicable.
      The method automatically determines whether to only run preprocessing,
      only evaluation (possibly on previously fitted preprocessing), or both.
      Calling ``fit`` will overwrite previously stored data where applicable.
  
      Parameters
      ----------
      X : array-like, shape=[n_samples, n_features]
          input data to preprocess and create folds from.
  
      y : array-like, shape=[n_samples, ]
          training labels.
  
      estimators : list or dict, optional
          set of estimators to use. If no preprocessing is desired or if
          only on preprocessing pipeline should apply to all, pass a list of
          estimators. The list can contain elements of named tuples
          (i.e. ``('my_name', my_est)``).
  
          If different estimators should be mapped to preprocessing cases,
          a dictionary that maps estimators to each case should
          be passed: ``{'case_a': list_of_est, ...}``.
  
      param_dicts : dict, optional
          parameter distribution mapping for estimators. Current
          implementation only supports randomized grid search. Passed
          distribution object must have an ``rvs`` method.
          See :mod:`scipy.stats` for details.
  
          There is quite some flexibility in specifying ``param_dicts``. If
          there is no preprocessing, or if all estimators are fitted on all
          preprocessing cases, the ``param_dict`` should have keys matching
          the names of the estimators. ::
  
              estimators = [('name', est), est]
  
              param_dicts = {'name': {'param-1': some_distribution},
                             'est': {'param-1': some_distribution}
                            }
  
          It is possible to specify different distributions for some or all
          preprocessing cases::
  
              preprocessing = {'case-1': transformer_list,
                               'case-2': transformer_list}
  
              estimators = [('name', est), est]
  
              param_dicts = {'name':
                                 {'param-1': some_distribution},
                             ('case-1', 'est'):
                                 {'param-1': some_distribution}
                             ('case-2', 'est'):
                                 {'param-1': some_distribution,
                                  'param-2': some_distribution}
                            }
  
          If estimators are mapped on a per-preprocessing case basis as a
          dictionary, ``param_dict`` must have key entries of the form
          ``(case_name, est_name)``.
  
      n_iter : int
          number of parameter draws to evaluate.
  
      preprocessing : dict, optional
          preprocessing cases to consider. Pass a dictionary mapping a
          case name to a preprocessing pipeline. ::
  
              preprocessing = {'case_name': transformer_list,}
  
      Returns
      -------
      self : instance
          class instance with stored estimator evaluation results in
          the ``results`` attribute.
      """
      job = set_job(estimators, preprocessing)
      self._initialize(
          job, estimators, preprocessing, param_dicts, n_iter
      )
      self._fit(X, y, job)
      self._get_results()
      return self
  
imports: from __future__ import division, with_statement
  import warnings
  import numpy
  from ._base_functions import parse_key, set_job, cat, check_scorer, make_learners, make_tansformers, check_instances
  from ..index import FoldIndex
  from ..parallel import ParallelEvaluation
  from ..parallel.base import BaseBackend, IndexMixin
  from ..metrics import Data, assemble_data
  from ..utils.formatting import _flatten, _check_instances
  from ..utils import print_time, safe_print, assert_correct_format
  from ..externals.joblib import delayed
  from ..externals.sklearn.base import clone
label_desc: Use image node_id 2 for calling the Evaluator obj's underlying member method code with example usage: obj.fit(X, y, estimators, param_dicts, n_iter, preprocessing) and returns: self