class_docstr: A Trainer for data parallel Mosaic Composers on PyTorch training.
  
  This Trainer runs the ``composer.trainer.Trainer.fit()`` method on multiple
  Ray Actors. The training is carried out in a distributed fashion through PyTorch
  DDP. These actors already have the necessary torch process group already
  configured for distributed PyTorch training.
  
  The training function ran on every Actor will first run the
  specified ``trainer_init_per_worker`` function to obtain an instantiated
  ``composer.Trainer`` object. The ``trainer_init_per_worker`` function
  will have access to preprocessed train and evaluation datasets.
  
  Example:
  
  ..
      TODO(yunxuan): Enable the test after we resolve the mosaicml dependency issue
  
  .. testcode::
      :skipif: True
  
      import torch.utils.data
      import torchvision
      from torchvision import transforms, datasets
  
      from composer.models.tasks import ComposerClassifier
      import composer.optim
      from composer.algorithms import LabelSmoothing
  
      import ray
      import ray.train as train
      from ray.train import ScalingConfig
      from ray.train.mosaic import MosaicTrainer
  
      def trainer_init_per_worker(config):
          # prepare the model for distributed training and wrap with
          # ComposerClassifier for Composer Trainer compatibility
          model = torchvision.models.resnet18(num_classes=10)
          model = ComposerClassifier(ray.train.torch.prepare_model(model))
  
          # prepare train/test dataset
          mean = (0.507, 0.487, 0.441)
          std = (0.267, 0.256, 0.276)
          cifar10_transforms = transforms.Compose(
              [transforms.ToTensor(), transforms.Normalize(mean, std)]
          )
          data_directory = "~/data"
          train_dataset = datasets.CIFAR10(
              data_directory,
              train=True,
              download=True,
              transform=cifar10_transforms
          )
  
          # prepare train dataloader
          batch_size_per_worker = BATCH_SIZE // session.get_world_size()
          train_dataloader = torch.utils.data.DataLoader(
              train_dataset,
              batch_size=batch_size_per_worker
          )
          train_dataloader = ray.train.torch.prepare_data_loader(train_dataloader)
  
          # prepare optimizer
          optimizer = composer.optim.DecoupledSGDW(
              model.parameters(),
              lr=0.05,
              momentum=0.9,
              weight_decay=2.0e-3,
          )
  
          return composer.trainer.Trainer(
              model=model,
              train_dataloader=train_dataloader,
              optimizers=optimizer,
              **config
          )
  
      scaling_config = ScalingConfig(num_workers=2, use_gpu=True)
      trainer_init_config = {
          "max_duration": "1ba",
          "algorithms": [LabelSmoothing()],
      }
  
      trainer = MosaicTrainer(
          trainer_init_per_worker=trainer_init_per_worker,
          trainer_init_config=trainer_init_config,
          scaling_config=scaling_config,
      )
  
      trainer.fit()
  
  .. testoutput::
          :hide:
  
          ...
  
  Args:
      trainer_init_per_worker: The function that returns an instantiated
          ``composer.Trainer`` object and takes in configuration
          dictionary (``config``) as an argument. This dictionary is based on
          ``trainer_init_config`` and is modified for Ray - Composer integration.
      datasets: Any Datasets to use for training. At the moment, we do not support
          passing datasets to the trainer and using the dataset shards in the trainer
          loop. Instead, configure and load the datasets inside
          ``trainer_init_per_worker`` function
      trainer_init_config: Configurations to pass into ``trainer_init_per_worker`` as
          kwargs. Although the kwargs can be hard-coded in the
          ``trainer_init_per_worker``, using the config allows the flexibility of
          reusing the same worker init function while changing the trainer arguments.
          For example, when hyperparameter tuning you can reuse the
          same ``trainer_init_per_worker`` function with different hyperparameter
          values rather than having multiple ``trainer_init_per_worker`` functions
          with different hard-coded hyperparameter values.
      torch_config: Configuration for setting up the PyTorch backend. If set to
          None, use the default configuration. This replaces the ``backend_config``
          arg of ``DataParallelTrainer``. Same as in ``TorchTrainer``.
      scaling_config: Configuration for how to scale data parallel training.
      dataset_config: Configuration for dataset ingest.
      run_config: Configuration for the execution of the training run.
      resume_from_checkpoint: A ``ray.train.Checkpoint`` to resume training from.
class_docstr_tok: ["A","Trainer","for","data","parallel","Mosaic","Composers","on","PyTorch","training",".","This","Trainer","runs","the","``","composer.trainer.Trainer.fit","(",")","``","method","on","multiple","Ray","Actors",".","The","training","is","carried","out","in","a","distributed","fashion","through","PyTorch","DDP",".","These","actors","already","have","the","necessary","torch","process","group","already","configured","for","distributed","PyTorch","training",".","The","training","function","ran","on","every","Actor","will","first","run","the","specified","``","trainer_init_per_worker","``","function","to","obtain","an","instantiated","``","composer.Trainer","``","object",".","The","``","trainer_init_per_worker","``","function","will","have","access","to","preprocessed","train","and","evaluation","datasets",".","Example",":","..","TODO","(","yunxuan",")",":","Enable","the","test","after","we","resolve","the","mosaicml","dependency","issue","..","testcode",":",":",":","skipif",":","True","import","torch.utils.data","import","torchvision","from","torchvision","import","transforms",",","datasets","from","composer.models.tasks","import","ComposerClassifier","import","composer.optim","from","composer.algorithms","import","LabelSmoothing","import","ray","import","ray.train","as","train","from","ray.train","import","ScalingConfig","from","ray.train.mosaic","import","MosaicTrainer","def","trainer_init_per_worker","(","config",")",":","#","prepare","the","model","for","distributed","training","and","wrap","with","#","ComposerClassifier","for","Composer","Trainer","compatibility","model","=","torchvision.models.resnet18","(","num_classes=10",")","model","=","ComposerClassifier","(","ray.train.torch.prepare_model","(","model",")",")","#","prepare","train\/test","dataset","mean","=","(","0.507",",","0.487",",","0.441",")","std","=","(","0.267",",","0.256",",","0.276",")","cifar10_transforms","=","transforms.Compose","(","[","transforms.ToTensor","(",")",",","transforms.Normalize","(","mean",",","std",")","]",")","data_directory","=","``","~\/data","''","train_dataset","=","datasets.CIFAR10","(","data_directory",",","train=True",",","download=True",",","transform=cifar10_transforms",")","#","prepare","train","dataloader","batch_size_per_worker","=","BATCH_SIZE","\/\/","session.get_world_size","(",")","train_dataloader","=","torch.utils.data.DataLoader","(","train_dataset",",","batch_size=batch_size_per_worker",")","train_dataloader","=","ray.train.torch.prepare_data_loader","(","train_dataloader",")","#","prepare","optimizer","optimizer","=","composer.optim.DecoupledSGDW","(","model.parameters","(",")",",","lr=0.05",",","momentum=0.9",",","weight_decay=2.0e-3",",",")","return","composer.trainer.Trainer","(","model=model",",","train_dataloader=train_dataloader",",","optimizers=optimizer",",","*","*","config",")","scaling_config","=","ScalingConfig","(","num_workers=2",",","use_gpu=True",")","trainer_init_config","=","{","``","max_duration","''",":","``","1ba","''",",","``","algorithms","''",":","[","LabelSmoothing","(",")","]",",","}","trainer","=","MosaicTrainer","(","trainer_init_per_worker=trainer_init_per_worker",",","trainer_init_config=trainer_init_config",",","scaling_config=scaling_config",",",")","trainer.fit","(",")","..","testoutput",":",":",":","hide",":","...","Args",":","trainer_init_per_worker",":","The","function","that","returns","an","instantiated","``","composer.Trainer","``","object","and","takes","in","configuration","dictionary","(","``","config","``",")","as","an","argument",".","This","dictionary","is","based","on","``","trainer_init_config","``","and","is","modified","for","Ray","-","Composer","integration",".","datasets",":","Any","Datasets","to","use","for","training",".","At","the","moment",",","we","do","not","support","passing","datasets","to","the","trainer","and","using","the","dataset","shards","in","the","trainer","loop",".","Instead",",","configure","and","load","the","datasets","inside","``","trainer_init_per_worker","``","function","trainer_init_config",":","Configurations","to","pass","into","``","trainer_init_per_worker","``","as","kwargs",".","Although","the","kwargs","can","be","hard-coded","in","the","``","trainer_init_per_worker","``",",","using","the","config","allows","the","flexibility","of","reusing","the","same","worker","init","function","while","changing","the","trainer","arguments",".","For","example",",","when","hyperparameter","tuning","you","can","reuse","the","same","``","trainer_init_per_worker","``","function","with","different","hyperparameter","values","rather","than","having","multiple","``","trainer_init_per_worker","``","functions","with","different","hard-coded","hyperparameter","values",".","torch_config",":","Configuration","for","setting","up","the","PyTorch","backend",".","If","set","to","None",",","use","the","default","configuration",".","This","replaces","the","``","backend_config","``","arg","of","``","DataParallelTrainer","``",".","Same","as","in","``","TorchTrainer","``",".","scaling_config",":","Configuration","for","how","to","scale","data","parallel","training",".","dataset_config",":","Configuration","for","dataset","ingest",".","run_config",":","Configuration","for","the","execution","of","the","training","run",".","resume_from_checkpoint",":","A","``","ray.train.Checkpoint","``","to","resume","training","from","."]
code: def restore(cls: Type["MosaicTrainer"], **kwargs) -> "MosaicTrainer":
      # TODO(ml-team): Reenable after Mosaic checkpointing is supported
      raise NotImplementedError
  
file_path: ray/python/ray/train/mosaic/mosaic_trainer.py
imports: import inspect
  import warnings
  from typing import TYPE_CHECKING, Any, Callable, Dict, Optional, Type
  from composer.loggers.logger_destination import LoggerDestination
  from composer.trainer import Trainer
  from ray.train import Checkpoint, DataConfig, RunConfig, ScalingConfig
  from ray.train.mosaic._mosaic_utils import RayLogger
  from ray.train.torch import TorchConfig, TorchTrainer
  from ray.train.trainer import GenDataset
  from ray.util import PublicAPI
label_desc: Use image node_id 3 for calling the MosaicTrainer obj's underlying member method code with example usage: obj.restore(cls) without return types