ch 2.0 does not support the previous versions. Once CUDA is installed, torch nightly can be installed using:

	

	Copied
pip install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cu117

Using accelerated transformers and torch.compile.
	
Accelerated Transformers implementation
PyTorch 2.0 includes an optimized and memory-efficient attention implementation through the torch.nn.functional.scaled_dot_product_attentio