

torch.backends.cudnn


torch.backends.mps


torch.backends.mkl


torch.backends.mkldnn


torch.backends.openmp


torch.backends.opt_einsum


torch.backends.xeon


torch.distributed


Backends


Basics


Initialization


Post-Initialization


Distributed Key-Value Store


Groups


Point-to-point communication


Synchronous and asynchronous collective operations


Collective functions


Profiling Collective Communication


Multi-GPU collective functions


Third-party backends


Launch utility


Spawn utility


Debugging "torch.distributed" applications


Logging


torch.distributed.algorithms.join


torch.distributed.elastic


Get Started


Documentation


torch.distributed.fsdp


torch.distributed.optim


torch.distributed.tensor.parallel


torch.distributed.checkpoint


torch.distributions


Score function


Pathwise derivative


Distribution


ExponentialFamily


Bernoulli


Beta


Binomial

