class_docstr: Delegate the whole training to PyTorch Lightning.
  
  Since the arguments passed to the initialization needs to be serialized, ``LightningModule``, ``Trainer`` or
  ``DataLoader`` in this file should be used. Another option is to hide dataloader in the Lightning module, in
  which case, dataloaders are not required for this class to work.
  
  Following the programming style of Lightning, metrics sent to NNI should be obtained from ``callback_metrics``
  in trainer. Two hooks are added at the end of validation epoch and the end of ``fit``, respectively. The metric name
  and type depend on the specific task.
  
  .. warning::
  
     The Lightning evaluator are stateful. If you try to use a previous Lightning evaluator,
     please note that the inner ``lightning_module`` and ``trainer`` will be reused.
  
  Parameters
  ----------
  lightning_module
      Lightning module that defines the training logic.
  trainer
      Lightning trainer that handles the training.
  train_dataloders
      Used in ``trainer.fit()``. A PyTorch DataLoader with training samples.
      If the ``lightning_module`` has a predefined train_dataloader method this will be skipped.
      It can be any types of dataloader supported by Lightning.
  val_dataloaders
      Used in ``trainer.fit()``. Either a single PyTorch Dataloader or a list of them, specifying validation samples.
      If the ``lightning_module`` has a predefined val_dataloaders method this will be skipped.
      It can be any types of dataloader supported by Lightning.
  datamodule
      Used in ``trainer.fit()``. See `Lightning DataModule <https://pytorch-lightning.readthedocs.io/en/stable/data/datamodule.html>`__.
  fit_kwargs
      Keyword arguments passed to ``trainer.fit()``.
  detect_interrupt
      Lightning has a `graceful shutdown <https://pytorch-lightning.readthedocs.io/en/stable/common/trainer.html>`__
      mechanism. It does not terminate the whole program (but only the training) when a KeyboardInterrupt is received.
      Setting this to ``True`` will raise the KeyboardInterrupt to the main process, so that the whole program can be terminated.
  
  Examples
  --------
  Users should define a Lightning module that inherits :class:`LightningModule`,
  and use :class:`Trainer` and :class:`DataLoader` from ```nni.nas.evaluator.pytorch``,
  and make them parameters of this evaluator::
  
      import nni
      from nni.nas.evaluator.pytorch.lightning import Lightning, LightningModule, Trainer, DataLoader
class_docstr_tok: ["Delegate","the","whole","training","to","PyTorch","Lightning",".","Since","the","arguments","passed","to","the","initialization","needs","to","be","serialized",",","``","LightningModule","``",",","``","Trainer","``","or","``","DataLoader","``","in","this","file","should","be","used",".","Another","option","is","to","hide","dataloader","in","the","Lightning","module",",","in","which","case",",","dataloaders","are","not","required","for","this","class","to","work",".","Following","the","programming","style","of","Lightning",",","metrics","sent","to","NNI","should","be","obtained","from","``","callback_metrics","``","in","trainer",".","Two","hooks","are","added","at","the","end","of","validation","epoch","and","the","end","of","``","fit","``",",","respectively",".","The","metric","name","and","type","depend","on","the","specific","task",".","..","warning",":",":","The","Lightning","evaluator","are","stateful",".","If","you","try","to","use","a","previous","Lightning","evaluator",",","please","note","that","the","inner","``","lightning_module","``","and","``","trainer","``","will","be","reused",".","Parameters","--","--","--","--","--","lightning_module","Lightning","module","that","defines","the","training","logic",".","trainer","Lightning","trainer","that","handles","the","training",".","train_dataloders","Used","in","``","trainer.fit","(",")","``",".","A","PyTorch","DataLoader","with","training","samples",".","If","the","``","lightning_module","``","has","a","predefined","train_dataloader","method","this","will","be","skipped",".","It","can","be","any","types","of","dataloader","supported","by","Lightning",".","val_dataloaders","Used","in","``","trainer.fit","(",")","``",".","Either","a","single","PyTorch","Dataloader","or","a","list","of","them",",","specifying","validation","samples",".","If","the","``","lightning_module","``","has","a","predefined","val_dataloaders","method","this","will","be","skipped",".","It","can","be","any","types","of","dataloader","supported","by","Lightning",".","datamodule","Used","in","``","trainer.fit","(",")","``",".","See","`","Lightning","DataModule","<","https",":","\/\/pytorch-lightning.readthedocs.io\/en\/stable\/data\/datamodule.html",">","`","__",".","fit_kwargs","Keyword","arguments","passed","to","``","trainer.fit","(",")","``",".","detect_interrupt","Lightning","has","a","`","graceful","shutdown","<","https",":","\/\/pytorch-lightning.readthedocs.io\/en\/stable\/common\/trainer.html",">","`","__","mechanism",".","It","does","not","terminate","the","whole","program","(","but","only","the","training",")","when","a","KeyboardInterrupt","is","received",".","Setting","this","to","``","True","``","will","raise","the","KeyboardInterrupt","to","the","main","process",",","so","that","the","whole","program","can","be","terminated",".","Examples","--","--","--","--","Users","should","define","a","Lightning","module","that","inherits",":","class",":","`","LightningModule","`",",","and","use",":","class",":","`","Trainer","`","and",":","class",":","`","DataLoader","`","from","``","`","nni.nas.evaluator.pytorch","``",",","and","make","them","parameters","of","this","evaluator",":",":","import","nni","from","nni.nas.evaluator.pytorch.lightning","import","Lightning",",","LightningModule",",","Trainer",",","DataLoader"]
docstr: Fit the model with provided dataloader, with Lightning trainer.
  If ``train_dataloaders`` is not provided, ``trainer.validate()`` will be called.
  
  Parameters
  ----------
  model
      The model to fit.
code: def evaluate(self, model):
      """
      Fit the model with provided dataloader, with Lightning trainer.
      If ``train_dataloaders`` is not provided, ``trainer.validate()`` will be called.
  
      Parameters
      ----------
      model
          The model to fit.
      """
      if self.is_mutable():
          raise RuntimeError(
              "Mutable evaluator must first be `freeze()` before evaluation."
          )
  
      self.module.set_model(model)
      if self.datamodule is not None:
          _logger.info(
              "Fit with datamodule. Train and valid dataloaders will be ignored."
          )
          rv = self.trainer.fit(
              self.module, self.datamodule, **self.fit_kwargs
          )
      elif (
          self.train_dataloaders is None
          and self.val_dataloaders is not None
      ):
          _logger.info(
              "Only validation dataloaders are available. Skip to validation."
          )
          rv = self.trainer.validate(
              self.module, self.val_dataloaders, **self.fit_kwargs
          )
      else:
          if self.val_dataloaders is None:
              _logger.warning(
                  "Validation dataloaders are missing. Safe to ignore this warning when using one-shot strategy."
              )
          rv = self.trainer.fit(
              self.module,
              self.train_dataloaders,
              self.val_dataloaders,
              **self.fit_kwargs,
          )
  
      if self.detect_interrupt:
          from pytorch_lightning.trainer.states import TrainerStatus
  
          if self.trainer.state.status == TrainerStatus.INTERRUPTED:
              _logger.warning(
                  "Trainer status is detected to be interrupted."
              )
              raise KeyboardInterrupt(
                  "Trainer status is detected to be interrupted."
              )
  
      return rv
  
imports: import logging
  import os
  import warnings
  from pathlib import Path
  from typing import Any, Dict, Union, Optional, List, Type
  import pytorch_lightning
  import torch.nn
  import torch.nn.functional
  import torch.optim
  import torchmetrics
  import torchmetrics.classification
  import torch.utils.data
  import nni
  from nni.common.serializer import is_traceable
  from nni.nas.evaluator import MutableEvaluator
label_desc: Use image node_id 2 for calling the Lightning obj's underlying member method code with example usage: obj.evaluate(model) and returns: rv