A few of the highlights include: 

Automatic mixed precision (AMP) training is now natively supported and a stable feature (See here for more details) - thanks for NVIDIA’s contributions; 
Native TensorPipe support now added for tensor-aware, point-to-point communication primitives built specifically for machine learning; 
Added support for complex tensors to the frontend API surface;
New profiling tools providing tensor-level memory consumption information;
Numerous improvements and new features for both distributed data parallel (DDP) training and the remote procedural call (RPC) packages.
