(Beta) Forward-mode Automatic Differentiation
Forward-mode AD allows the computation of directional derivatives (or equivalently, Jacobian-vector products) eagerly in the forward pass. PyTorch 1.12 significantly improves the operator coverage for forward-mode AD. See our tutorial for more information.
TorchData
BC DataLoader + DataPipe
`DataPipe` from TorchData becomes fully backward compatible with the existing `DataLoader` regarding shuffle determinism and dynamic sharding in both multiprocessing and distributed environments. For more details, please check out the tutorial.
(Beta) AWS S3 Integration
DataPipes based on AWSSDK have been integrated into TorchData. It provides the following features backed by native AWSSDK: