class_docstr: This is a DeepChem model implemented by a PyTorch model.
  
  Here is a simple example of code that uses TorchModel to train
  a PyTorch model on a DeepChem dataset.
  
  >>> import torch
  >>> import deepchem as dc
  >>> import numpy as np
  >>> X, y = np.random.random((10, 100)), np.random.random((10, 1))
  >>> dataset = dc.data.NumpyDataset(X=X, y=y)
  >>> pytorch_model = torch.nn.Sequential(
  ...   torch.nn.Linear(100, 1000),
  ...   torch.nn.Tanh(),
  ...   torch.nn.Linear(1000, 1))
  >>> model = dc.models.TorchModel(pytorch_model, loss=dc.models.losses.L2Loss())
  >>> loss = model.fit(dataset, nb_epoch=5)
  
  The loss function for a model can be defined in two different
  ways.  For models that have only a single output and use a
  standard loss function, you can simply provide a
  dc.models.losses.Loss object.  This defines the loss for each
  sample or sample/task pair.  The result is automatically
  multiplied by the weights and averaged over the batch.
  
  For more complicated cases, you can instead provide a function
  that directly computes the total loss.  It must be of the form
  f(outputs, labels, weights), taking the list of outputs from
  the model, the expected values, and any weight matrices.  It
  should return a scalar equal to the value of the loss function
  for the batch.  No additional processing is done to the
  result; it is up to you to do any weighting, averaging, adding
  of penalty terms, etc.
  
  You can optionally provide an output_types argument, which
  describes how to interpret the model's outputs.  This should
  be a list of strings, one for each output. You can use an
  arbitrary output_type for a output, but some output_types are
  special and will undergo extra processing:
  
  - 'prediction': This is a normal output, and will be returned by predict().
      If output types are not specified, all outputs are assumed
      to be of this type.
  
  - 'loss': This output will be used in place of the normal
      outputs for computing the loss function.  For example,
      models that output probability distributions usually do it
      by computing unbounded numbers (the logits), then passing
      them through a softmax function to turn them into
      probabilities.  When computing the cross entropy, it is more
      numerically stable to use the logits directly rather than
      the probabilities.  You can do this by having the model
      produce both probabilities and logits as outputs, then
      specifying output_types=['prediction', 'loss'].  When
      predict() is called, only the first output (the
      probabilities) will be returned.  But during training, it is
      the second output (the logits) that will be passed to the
      loss function.
  
  - 'variance': This output is used for estimating the
      uncertainty in another output.  To create a model that can
      estimate uncertainty, there must be the same number of
      'prediction' and 'variance' outputs.  Each variance output
      must have the same shape as the corresponding prediction
      output, and each element is an estimate of the variance in
      the corresponding prediction.  Also be aware that if a model
      supports uncertainty, it MUST use dropout on every layer,
      and dropout most be enabled during uncertainty prediction.
      Otherwise, the uncertainties it computes will be inaccurate.
  
  - other: Arbitrary output_types can be used to extract outputs
      produced by the model, but will have no additional
      processing performed.
class_docstr_tok: ["This","is","a","DeepChem","model","implemented","by","a","PyTorch","model",".","Here","is","a","simple","example","of","code","that","uses","TorchModel","to","train","a","PyTorch","model","on","a","DeepChem","dataset",".",">",">",">","import","torch",">",">",">","import","deepchem","as","dc",">",">",">","import","numpy","as","np",">",">",">","X",",","y","=","np.random.random","(","(","10",",","100",")",")",",","np.random.random","(","(","10",",","1",")",")",">",">",">","dataset","=","dc.data.NumpyDataset","(","X=X",",","y=y",")",">",">",">","pytorch_model","=","torch.nn.Sequential","(","...","torch.nn.Linear","(","100",",","1000",")",",","...","torch.nn.Tanh","(",")",",","...","torch.nn.Linear","(","1000",",","1",")",")",">",">",">","model","=","dc.models.TorchModel","(","pytorch_model",",","loss=dc.models.losses.L2Loss","(",")",")",">",">",">","loss","=","model.fit","(","dataset",",","nb_epoch=5",")","The","loss","function","for","a","model","can","be","defined","in","two","different","ways",".","For","models","that","have","only","a","single","output","and","use","a","standard","loss","function",",","you","can","simply","provide","a","dc.models.losses.Loss","object",".","This","defines","the","loss","for","each","sample","or","sample\/task","pair",".","The","result","is","automatically","multiplied","by","the","weights","and","averaged","over","the","batch",".","For","more","complicated","cases",",","you","can","instead","provide","a","function","that","directly","computes","the","total","loss",".","It","must","be","of","the","form","f","(","outputs",",","labels",",","weights",")",",","taking","the","list","of","outputs","from","the","model",",","the","expected","values",",","and","any","weight","matrices",".","It","should","return","a","scalar","equal","to","the","value","of","the","loss","function","for","the","batch",".","No","additional","processing","is","done","to","the","result",";","it","is","up","to","you","to","do","any","weighting",",","averaging",",","adding","of","penalty","terms",",","etc",".","You","can","optionally","provide","an","output_types","argument",",","which","describes","how","to","interpret","the","model","'s","outputs",".","This","should","be","a","list","of","strings",",","one","for","each","output",".","You","can","use","an","arbitrary","output_type","for","a","output",",","but","some","output_types","are","special","and","will","undergo","extra","processing",":","-","'prediction","'",":","This","is","a","normal","output",",","and","will","be","returned","by","predict","(",")",".","If","output","types","are","not","specified",",","all","outputs","are","assumed","to","be","of","this","type",".","-","'loss","'",":","This","output","will","be","used","in","place","of","the","normal","outputs","for","computing","the","loss","function",".","For","example",",","models","that","output","probability","distributions","usually","do","it","by","computing","unbounded","numbers","(","the","logits",")",",","then","passing","them","through","a","softmax","function","to","turn","them","into","probabilities",".","When","computing","the","cross","entropy",",","it","is","more","numerically","stable","to","use","the","logits","directly","rather","than","the","probabilities",".","You","can","do","this","by","having","the","model","produce","both","probabilities","and","logits","as","outputs",",","then","specifying","output_types=","[","'prediction","'",",","'loss","'","]",".","When","predict","(",")","is","called",",","only","the","first","output","(","the","probabilities",")","will","be","returned",".","But","during","training",",","it","is","the","second","output","(","the","logits",")","that","will","be","passed","to","the","loss","function",".","-","'variance","'",":","This","output","is","used","for","estimating","the","uncertainty","in","another","output",".","To","create","a","model","that","can","estimate","uncertainty",",","there","must","be","the","same","number","of","'prediction","'","and","'variance","'","outputs",".","Each","variance","output","must","have","the","same","shape","as","the","corresponding","prediction","output",",","and","each","element","is","an","estimate","of","the","variance","in","the","corresponding","prediction",".","Also","be","aware","that","if","a","model","supports","uncertainty",",","it","MUST","use","dropout","on","every","layer",",","and","dropout","most","be","enabled","during","uncertainty","prediction",".","Otherwise",",","the","uncertainties","it","computes","will","be","inaccurate",".","-","other",":","Arbitrary","output_types","can","be","used","to","extract","outputs","produced","by","the","model",",","but","will","have","no","additional","processing","performed","."]
docstr: Train this model on data from a generator.
  
  Parameters
  ----------
  generator: generator
      this should generate batches, each represented as a tuple of the form
      (inputs, labels, weights).
  max_checkpoints_to_keep: int
      the maximum number of checkpoints to keep.  Older checkpoints are discarded.
  checkpoint_interval: int
      the frequency at which to write checkpoints, measured in training steps.
      Set this to 0 to disable automatic checkpointing.
  restore: bool
      if True, restore the model from the most recent checkpoint and continue training
      from there.  If False, retrain the model from scratch.
  variables: list of torch.nn.Parameter
      the variables to train.  If None (the default), all trainable variables in
      the model are used.
  loss: function
      a function of the form f(outputs, labels, weights) that computes the loss
      for each batch.  If None (the default), the model's standard loss function
      is used.
  callbacks: function or list of functions
      one or more functions of the form f(model, step) that will be invoked after
      every step.  This can be used to perform validation, logging, etc.
  all_losses: Optional[List[float]], optional (default None)
      If specified, all logged losses are appended into this list. Note that
      you can call `fit()` repeatedly with the same list and losses will
      continue to be appended.
  
  Returns
  -------
  The average loss over the most recent checkpoint interval
docstr_tok: ["Train","this","model","on","data","from","a","generator",".","Parameters","--","--","--","--","--","generator",":","generator","this","should","generate","batches",",","each","represented","as","a","tuple","of","the","form","(","inputs",",","labels",",","weights",")",".","max_checkpoints_to_keep",":","int","the","maximum","number","of","checkpoints","to","keep",".","Older","checkpoints","are","discarded",".","checkpoint_interval",":","int","the","frequency","at","which","to","write","checkpoints",",","measured","in","training","steps",".","Set","this","to","0","to","disable","automatic","checkpointing",".","restore",":","bool","if","True",",","restore","the","model","from","the","most","recent","checkpoint","and","continue","training","from","there",".","If","False",",","retrain","the","model","from","scratch",".","variables",":","list","of","torch.nn.Parameter","the","variables","to","train",".","If","None","(","the","default",")",",","all","trainable","variables","in","the","model","are","used",".","loss",":","function","a","function","of","the","form","f","(","outputs",",","labels",",","weights",")","that","computes","the","loss","for","each","batch",".","If","None","(","the","default",")",",","the","model","'s","standard","loss","function","is","used",".","callbacks",":","function","or","list","of","functions","one","or","more","functions","of","the","form","f","(","model",",","step",")","that","will","be","invoked","after","every","step",".","This","can","be","used","to","perform","validation",",","logging",",","etc",".","all_losses",":","Optional","[","List","[","float","]","]",",","optional","(","default","None",")","If","specified",",","all","logged","losses","are","appended","into","this","list",".","Note","that","you","can","call","`","fit","(",")","`","repeatedly","with","the","same","list","and","losses","will","continue","to","be","appended",".","Returns","--","--","--","-","The","average","loss","over","the","most","recent","checkpoint","interval"]
code: def fit_generator(
      self,
      generator: Iterable[Tuple[Any, Any, Any]],
      max_checkpoints_to_keep: int = 5,
      checkpoint_interval: int = 1000,
      restore: bool = False,
      variables: Optional[List[torch.nn.Parameter]] = None,
      loss: Optional[LossFn] = None,
      callbacks: Union[Callable, List[Callable]] = [],
      all_losses: Optional[List[float]] = None,
  ) -> float:
      """Train this model on data from a generator.
  
      Parameters
      ----------
      generator: generator
          this should generate batches, each represented as a tuple of the form
          (inputs, labels, weights).
      max_checkpoints_to_keep: int
          the maximum number of checkpoints to keep.  Older checkpoints are discarded.
      checkpoint_interval: int
          the frequency at which to write checkpoints, measured in training steps.
          Set this to 0 to disable automatic checkpointing.
      restore: bool
          if True, restore the model from the most recent checkpoint and continue training
          from there.  If False, retrain the model from scratch.
      variables: list of torch.nn.Parameter
          the variables to train.  If None (the default), all trainable variables in
          the model are used.
      loss: function
          a function of the form f(outputs, labels, weights) that computes the loss
          for each batch.  If None (the default), the model's standard loss function
          is used.
      callbacks: function or list of functions
          one or more functions of the form f(model, step) that will be invoked after
          every step.  This can be used to perform validation, logging, etc.
      all_losses: Optional[List[float]], optional (default None)
          If specified, all logged losses are appended into this list. Note that
          you can call `fit()` repeatedly with the same list and losses will
          continue to be appended.
  
      Returns
      -------
      The average loss over the most recent checkpoint interval
      """
      if not isinstance(callbacks, SequenceCollection):
          callbacks = [callbacks]
      self._ensure_built()
      self.model.train()
      avg_loss = 0.0
      last_avg_loss = 0.0
      averaged_batches = 0
      if loss is None:
          loss = self._loss_fn
      if variables is None:
          optimizer = self._pytorch_optimizer
          lr_schedule = self._lr_schedule
      else:
          var_key = tuple(variables)
          if var_key in self._optimizer_for_vars:
              optimizer, lr_schedule = self._optimizer_for_vars[var_key]
          else:
              optimizer = self.optimizer._create_pytorch_optimizer(
                  variables
              )
              if isinstance(
                  self.optimizer.learning_rate, LearningRateSchedule
              ):
                  lr_schedule = self.optimizer.learning_rate._create_pytorch_schedule(
                      optimizer
                  )
              else:
                  lr_schedule = None
              self._optimizer_for_vars[var_key] = (
                  optimizer,
                  lr_schedule,
              )
      time1 = time.time()
  
      # Main training loop.
  
      for batch in generator:
          if restore:
              self.restore()
              restore = False
          inputs: OneOrMany[torch.Tensor]
          inputs, labels, weights = self._prepare_batch(batch)
  
          # Execute the loss function, accumulating the gradients.
  
          if isinstance(inputs, list) and len(inputs) == 1:
              inputs = inputs[0]
  
          optimizer.zero_grad()
          outputs = self.model(inputs)
          if isinstance(outputs, torch.Tensor):
              outputs = [outputs]
          if self._loss_outputs is not None:
              outputs = [outputs[i] for i in self._loss_outputs]
          batch_loss = loss(outputs, labels, weights)
          batch_loss.backward()
          optimizer.step()
          if lr_schedule is not None:
              lr_schedule.step()
          self._global_step += 1
          current_step = self._global_step
  
          avg_loss += batch_loss
  
          # Report progress and write checkpoints.
          averaged_batches += 1
          should_log = current_step % self.log_frequency == 0
          if should_log:
              avg_loss = float(avg_loss) / averaged_batches
              logger.info(
                  "Ending global_step %d: Average loss %g"
                  % (current_step, avg_loss)
              )
              if all_losses is not None:
                  all_losses.append(avg_loss)
              # Capture the last avg_loss in case of return since we're resetting to 0 now
              last_avg_loss = avg_loss
              avg_loss = 0.0
              averaged_batches = 0
  
          if (
              checkpoint_interval > 0
              and current_step % checkpoint_interval
              == checkpoint_interval - 1
          ):
              self.save_checkpoint(max_checkpoints_to_keep)
          for c in callbacks:
              c(self, current_step)
          if self.tensorboard and should_log:
              self._log_scalar_to_tensorboard(
                  "loss", batch_loss, current_step
              )
          if (self.wandb_logger is not None) and should_log:
              all_data = dict({"train/loss": batch_loss})
              self.wandb_logger.log_data(all_data, step=current_step)
  
      # Report final results.
      if averaged_batches > 0:
          avg_loss = float(avg_loss) / averaged_batches
          logger.info(
              "Ending global_step %d: Average loss %g"
              % (current_step, avg_loss)
          )
          if all_losses is not None:
              all_losses.append(avg_loss)
          last_avg_loss = avg_loss
  
      if checkpoint_interval > 0:
          self.save_checkpoint(max_checkpoints_to_keep)
  
      time2 = time.time()
      logger.info(
          "TIMING: model fitting took %0.3f s" % (time2 - time1)
      )
      return last_avg_loss
  
imports: import numpy
  import torch
  import time
  import logging
  import os
  import datetime
  from deepchem.data import Dataset, NumpyDataset
  from deepchem.metrics import Metric
  from deepchem.models.losses import Loss
  from deepchem.models.models import Model
  from deepchem.models.optimizers import Adam, Optimizer, LearningRateSchedule
  from deepchem.trans import Transformer, undo_transforms
  from deepchem.utils.evaluate import GeneratorEvaluator
  from collections.abc import Sequence
  from typing import Any, Callable, Dict, Iterable, List, Optional, Sequence, Tuple, Union
  from deepchem.utils.typing import LossFn, OneOrMany
  from deepchem.models.wandblogger import WandbLogger
label_desc: Use image node_id 4 for calling the TorchModel obj's underlying member method code with example usage: obj.fit_generator(generator, max_checkpoints_to_keep, checkpoint_interval, restore, variables, loss, callbacks, all_losses) and returns: last_avg_loss