scaled by the same factor.  In other words, gradient values have a
larger magnitude, so they don't flush to zero.
Each parameter's gradient (".grad" attribute) should be unscaled
before the optimizer updates the parameters, so the scale factor does
not interfere with the learning rate.
class torch.cuda.amp.GradScaler(init_scale=65536.0, growth_factor=2.0, backoff_factor=0.5, growth_interval=2000, enabled=True)
get_backoff_factor()
  Returns a Python float containing the scale backoff factor.

get_growth_factor()
  Returns a Python float containing the scale growth factor.

get_growth_interval()
  Returns a Python int containing the growth interval.

get_scale()
  Returns a Python float containing the current scale, or 1.0 if
  scaling is disabled.

  Warning:

    "get_scale()" incurs a CPU-GPU sync.

is_enabled()
  Returns a bool indicating whether this instance is enabled.

load_state_dict(state_dict)