class_docstr: FP16 Optimizer for training fp16 models. Handles loss scaling.
  
  For usage example please see, TODO:  DeepSpeed V2 Tutorial
   
class_docstr_tok: ["FP16","Optimizer","for","training","fp16","models",".","Handles","loss","scaling",".","For","usage","example","please","see",",","TODO",":","DeepSpeed","V2","Tutorial"]
docstr: :attr:`backward` performs the following steps:
  
  1. fp32_loss = loss.float()
  2. scaled_loss = fp32_loss*loss_scale
  3. scaled_loss.backward(), which accumulates scaled gradients into the ``.grad`` attributes of the model's fp16 leaves
docstr_tok: [":","attr",":","`","backward","`","performs","the","following","steps",":","1.","fp32_loss","=","loss.float","(",")","2.","scaled_loss","=","fp32_loss","*","loss_scale","3.","scaled_loss.backward","(",")",",","which","accumulates","scaled","gradients","into","the","``",".grad","``","attributes","of","the","model","'s","fp16","leaves"]
code: def backward(self, loss, create_graph=False, retain_graph=False):
      """
      :attr:`backward` performs the following steps:
  
      1. fp32_loss = loss.float()
      2. scaled_loss = fp32_loss*loss_scale
      3. scaled_loss.backward(), which accumulates scaled gradients into the ``.grad`` attributes of the model's fp16 leaves
      """
      if self.custom_loss_scaler:
          scaled_loss = self.external_loss_scale * loss
          scaled_loss.backward()
      else:
          scaled_loss = (loss.float()) * self.cur_scale
          scaled_loss.backward(
              create_graph=create_graph, retain_graph=retain_graph
          )
  
imports: import torch
  from torch._utils import _flatten_dense_tensors, _unflatten_dense_tensors
  from deepspeed.runtime import DeepSpeedOptimizer
  from deepspeed.runtime.utils import get_global_norm, get_grad_norm, CheckOverflow, get_weight_norm, required_torch_version
  from deepspeed.runtime.fp16.loss_scaler import INITIAL_LOSS_SCALE, SCALE_WINDOW, MIN_LOSS_SCALE
  from deepspeed.utils import groups, logger, log_dist
  from deepspeed import comm
  from deepspeed.checkpoint.constants import OPTIMIZER_STATE_DICT, CLIP_GRAD
  from deepspeed.accelerator import get_accelerator
label_desc: Use image node_id 11 for calling the FP16_Optimizer obj's underlying member method code with example usage: obj.backward(loss, create_graph, retain_graph) without return types