class_docstr: Evaluate models on a stream of data.
  
  This class is a partner class to `Evaluator`. Instead of operating
  over datasets this class operates over a generator which yields
  batches of data to feed into provided model.
  
  Examples
  --------
  >>> import deepchem as dc
  >>> import numpy as np
  >>> X = np.random.rand(10, 5)
  >>> y = np.random.rand(10, 1)
  >>> dataset = dc.data.NumpyDataset(X, y)
  >>> model = dc.models.MultitaskRegressor(1, 5)
  >>> generator = model.default_generator(dataset, pad_batches=False)
  >>> transformers = []
  
  Then you can evaluate this model as follows
  
  >>> import sklearn
  >>> evaluator = GeneratorEvaluator(model, generator, transformers)
  >>> multitask_scores = evaluator.compute_model_performance(
  ...     sklearn.metrics.mean_absolute_error)
  
  Evaluators can also be used with `dc.metrics.Metric` objects as well
  in case you want to customize your metric further. (Note that a given
  generator can only be used once so we have to redefine the generator here.)
  
  >>> generator = model.default_generator(dataset, pad_batches=False)
  >>> evaluator = GeneratorEvaluator(model, generator, transformers)
  >>> metric = dc.metrics.Metric(dc.metrics.mae_score)
  >>> multitask_scores = evaluator.compute_model_performance(metric)
class_docstr_tok: ["Evaluate","models","on","a","stream","of","data",".","This","class","is","a","partner","class","to","`","Evaluator","`",".","Instead","of","operating","over","datasets","this","class","operates","over","a","generator","which","yields","batches","of","data","to","feed","into","provided","model",".","Examples","--","--","--","--",">",">",">","import","deepchem","as","dc",">",">",">","import","numpy","as","np",">",">",">","X","=","np.random.rand","(","10",",","5",")",">",">",">","y","=","np.random.rand","(","10",",","1",")",">",">",">","dataset","=","dc.data.NumpyDataset","(","X",",","y",")",">",">",">","model","=","dc.models.MultitaskRegressor","(","1",",","5",")",">",">",">","generator","=","model.default_generator","(","dataset",",","pad_batches=False",")",">",">",">","transformers","=","[","]","Then","you","can","evaluate","this","model","as","follows",">",">",">","import","sklearn",">",">",">","evaluator","=","GeneratorEvaluator","(","model",",","generator",",","transformers",")",">",">",">","multitask_scores","=","evaluator.compute_model_performance","(","...","sklearn.metrics.mean_absolute_error",")","Evaluators","can","also","be","used","with","`","dc.metrics.Metric","`","objects","as","well","in","case","you","want","to","customize","your","metric","further",".","(","Note","that","a","given","generator","can","only","be","used","once","so","we","have","to","redefine","the","generator","here",".",")",">",">",">","generator","=","model.default_generator","(","dataset",",","pad_batches=False",")",">",">",">","evaluator","=","GeneratorEvaluator","(","model",",","generator",",","transformers",")",">",">",">","metric","=","dc.metrics.Metric","(","dc.metrics.mae_score",")",">",">",">","multitask_scores","=","evaluator.compute_model_performance","(","metric",")"]
docstr: Parameters
  ----------
  model: Model
      Model to evaluate.
  generator: generator
      Generator which yields batches to feed into the model. For a
      KerasModel, it should be a tuple of the form (inputs, labels,
      weights). The "correct" way to create this generator is to use
      `model.default_generator` as shown in the example above.
  transformers: List[Transformer]
      Tranformers to "undo" when applied to the models outputs
  labels: list of Layer
      layers which are keys in the generator to compare to outputs
  weights: list of Layer
      layers which are keys in the generator for weight matrices
docstr_tok: ["Parameters","--","--","--","--","--","model",":","Model","Model","to","evaluate",".","generator",":","generator","Generator","which","yields","batches","to","feed","into","the","model",".","For","a","KerasModel",",","it","should","be","a","tuple","of","the","form","(","inputs",",","labels",",","weights",")",".","The","``","correct","''","way","to","create","this","generator","is","to","use","`","model.default_generator","`","as","shown","in","the","example","above",".","transformers",":","List","[","Transformer","]","Tranformers","to","``","undo","''","when","applied","to","the","models","outputs","labels",":","list","of","Layer","layers","which","are","keys","in","the","generator","to","compare","to","outputs","weights",":","list","of","Layer","layers","which","are","keys","in","the","generator","for","weight","matrices"]
code: def __init__(
      self,
      model,
      generator: Iterable[Tuple[Any, Any, Any]],
      transformers: List["dc.trans.Transformer"],
      labels: Optional[List] = None,
      weights: Optional[List] = None,
  ):
      """
      Parameters
      ----------
      model: Model
          Model to evaluate.
      generator: generator
          Generator which yields batches to feed into the model. For a
          KerasModel, it should be a tuple of the form (inputs, labels,
          weights). The "correct" way to create this generator is to use
          `model.default_generator` as shown in the example above.
      transformers: List[Transformer]
          Tranformers to "undo" when applied to the models outputs
      labels: list of Layer
          layers which are keys in the generator to compare to outputs
      weights: list of Layer
          layers which are keys in the generator for weight matrices
      """
      self.model = model
      self.generator = generator
      self.output_transformers = [
          transformer
          for transformer in transformers
          if transformer.transform_y
      ]
      self.label_keys = labels
      self.weights = weights
      if labels is not None and len(labels) != 1:
          raise ValueError(
              "GeneratorEvaluator currently only supports one label"
          )
  
imports: import csv
  import logging
  import warnings
  from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, Union
  import numpy
  import deepchem
  from deepchem.metrics import Metric
label_desc: Use image node_id 1 to create a new GeneratorEvaluator object from inherited base classes: object with example: obj = GeneratorEvaluator(model, generator, transformers, labels, weights)