How to downgrade from CUDA 11.4 to 10.2 & add sm_35 - CUDA error: no kernel image is available for execution on the device
I'm trying to run a piece of code on Pytorch, but I get the error:
RuntimeError: CUDA error: no kernel image is available for execution on the device

I've narrowed down the issue to be a missmatch of CUDA versions. My machine has 2 GPUs:
a GeForce GTX 650 (compute capability 3.0) and a Tesla K40c (compute capability 3.5). I've checked the compute capabilities here: https://developer.nvidia.com/cuda-gpus.
My nvidia-smi command gives the following:
nvidia-smi output (Driver Version: 470.57.02 &amp; CUDA Version: 11.4)
While my nvcc -V command gives the following:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243

The 10.1 version exists because I tried to install that CUDA version, specifically, following the instructions elsewhere (for example: https://medium.com/@anarmammadli/how-to-install-cuda-10-2-cudnn-7-6-5-and-samples-on-ubuntu-18-04-2493124478ca)
Also, I've installed cudatoolkit with conda, and so on my conda list I have the following entry:
...
cudatoolkit               10.1.243             h6bb024c_0
...

In accordance with https://github.com/moi90/pytorch_compute_capabilities/blob/main/table.md I've also installed the 1.8.0 PyTorch version.
However, in Python 3.7.11:
Python 3.7.11 (default, Jul 27 2021, 14:32:16)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.
&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.8.1'
&gt;&gt;&gt; torch.version.cuda
'10.1'
&gt;&gt;&gt; torch.cuda.get_arch_list()
['sm_37', 'sm_50', 'sm_60', 'sm_61', 'sm_70', 'sm_75', 'compute_37']
&gt;&gt;&gt; torch.cuda.is_available()
True

I don't have sm_35 available, which I need to use the Tesla K40. I believe this is the reason why I keep getting the CUDA error: no kernel image is available for execution on the device error. I've also tried all the above for version 10.2 of CUDA, same result.

I've solved my problem. As stated in the comments, I required a version of PyTorch that supports sm_35 compute capability. It had little to do with the current CUDA version. In the end, I found these binaries:
https://blog.nelsonliu.me/2020/10/13/newer-pytorch-binaries-for-older-gpus/
I finally fixed the issue by creating a new environment and running:
pip install torch==1.3.1+cu92 -f https://nelsonliu.me/files/pytorch/whl/torch_stable.html

