
No need to build extensions
Windows support
Bitwise accurate saving/restoring of checkpoints
DataParallel and intra-process model parallelism (although we still recommend torch.nn.DistributedDataParallel with one GPU per process as the most performant approach)
Gradient penalty (double backward)
