torch.mm again runs in float16 and produces float16 output, regardless of input types.
      g_float16 = torch.mm(d_float32, f_float32)

The autocast state is thread-local.  If you want it enabled in a
   new thread, the context manager or decorator must be invoked in
   that thread.  This affects "torch.nn.DataParallel" and
   "torch.nn.parallel.DistributedDataParallel" when used with more
   than one GPU per process (see Working with Multiple GPUs).
Parameters:
      * device_type (str, required) -- Whether to use 'cuda'
        or 'cpu' device
  * **enabled** (*bool**, **optional*) -- Whether autocasting
    should be enabled in the region. Default: "True"

  * **dtype** (*torch_dtype**, **optional*) -- Whether to use
    torch.float16 or torch.bfloat16.

  * **cache_enabled** (*bool**, **optional*) -- Whether the weight
    cache inside autocast should be enabled. Default: "True"
