RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. GPU not detected by pytorch
Having trouble with CUDA + Pytorch this is the error. I reinstalled CUDA and cudnn multiple times.
Conda env is detecting GPU but its giving errors with pytorch and certain cuda libraries. I tried with Cuda 10.1 and 10.0, and cudnn version 8 and 7.6.5, Added cuda to path and everything.
However anaconda is showing cuda tool kit 9.0 is installed, whilst I clearly installed 10.0, so I am not entirely sure what's the deal with that.
    =&gt; loading model from models/pytorch/pose_coco/pose_hrnet_w32_256x192.pth
Traceback (most recent call last):
  File &quot;hydroman2.py&quot;, line 580, in &lt;module&gt;
    pose_model.load_state_dict(torch.load(cfg.TEST.MODEL_FILE), strict=False)
  File &quot;C:\Users\Fardin\anaconda3\envs\myenv\lib\site-packages\torch\serialization.py&quot;, line 593, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File &quot;C:\Users\Fardin\anaconda3\envs\myenv\lib\site-packages\torch\serialization.py&quot;, line 773, in _legacy_load
    result = unpickler.load()
  File &quot;C:\Users\Fardin\anaconda3\envs\myenv\lib\site-packages\torch\serialization.py&quot;, line 729, in persistent_load
    deserialized_objects[root_key] = restore_location(obj, location)
  File &quot;C:\Users\Fardin\anaconda3\envs\myenv\lib\site-packages\torch\serialization.py&quot;, line 178, in default_restore_location
    result = fn(storage, location)
  File &quot;C:\Users\Fardin\anaconda3\envs\myenv\lib\site-packages\torch\serialization.py&quot;, line 154, in _cuda_deserialize
    device = validate_cuda_device(location)
  File &quot;C:\Users\Fardin\anaconda3\envs\myenv\lib\site-packages\torch\serialization.py&quot;, line 138, in validate_cuda_device
    raise RuntimeError('Attempting to deserialize object on a CUDA '
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.

System info
   System info:
--------------------------------------------------------------------------------
__Time Stamp__
Report started (local time)                   : 2021-03-19 19:59:06.957967
UTC start time                                : 2021-03-19 15:59:06.957967
Running time (s)                              : 4.003899

__Hardware Information__
Machine                                       : AMD64
CPU Name                                      : znver1
CPU Count                                     : 12
Number of accessible CPUs                     : 12
List of accessible CPUs cores                 : 0 1 2 3 4 5 6 7 8 9 10 11
CFS Restrictions (CPUs worth of runtime)      : None

CPU Features                                  : 64bit adx aes avx avx2 bmi bmi2
                                                clflushopt clzero cmov cx16 cx8
                                                f16c fma fsgsbase fxsr lzcnt mmx
                                                movbe mwaitx pclmul popcnt prfchw
                                                rdrnd rdseed sahf sha sse sse2
                                                sse3 sse4.1 sse4.2 sse4a ssse3
                                                xsave xsavec xsaveopt xsaves

Memory Total (MB)                             : 16334
Memory Available (MB)                         : 8787

__OS Information__
Platform Name                                 : Windows-10-10.0.19041-SP0
Platform Release                              : 10
OS Name                                       : Windows
OS Version                                    : 10.0.19041
OS Specific Version                           : 10 10.0.19041 SP0 Multiprocessor Free
Libc Version                                  : ?

__Python Information__
Python Compiler                               : MSC v.1916 64 bit (AMD64)
Python Implementation                         : CPython
Python Version                                : 3.8.5
Python Locale                                 : en_US.cp1252

__LLVM Information__
LLVM Version                                  : 10.0.1

__CUDA Information__
CUDA Device Initialized                       : True
CUDA Driver Version                           : 11020
CUDA Detect Output:
Found 1 CUDA devices
id 0     b'GeForce GTX 1070'                              [SUPPORTED]
                      compute capability: 6.1
                           pci device id: 0
                              pci bus id: 6
Summary:
        1/1 devices are supported

CUDA Librairies Test Output:
Finding cublas from &lt;unknown&gt;
        named  cublas.dll
        trying to open library...       ERROR: failed to open cublas:
Could not find module 'cublas.dll' (or one of its dependencies). Try using the full path with constructor syntax.
Finding cusparse from &lt;unknown&gt;
        named  cusparse.dll
        trying to open library...       ERROR: failed to open cusparse:
Could not find module 'cusparse.dll' (or one of its dependencies). Try using the full path with constructor syntax.
Finding cufft from &lt;unknown&gt;
        named  cufft.dll
        trying to open library...       ERROR: failed to open cufft:
Could not find module 'cufft.dll' (or one of its dependencies). Try using the full path with constructor syntax.
Finding curand from &lt;unknown&gt;
        named  curand.dll
        trying to open library...       ERROR: failed to open curand:
Could not find module 'curand.dll' (or one of its dependencies). Try using the full path with constructor syntax.
Finding nvvm from &lt;unknown&gt;
        named  nvvm.dll
        trying to open library...       ERROR: failed to open nvvm:
Could not find module 'nvvm.dll' (or one of its dependencies). Try using the full path with constructor syntax.
Finding cudart from &lt;unknown&gt;
        named  cudart.dll
        trying to open library...       ERROR: failed to open cudart:
Could not find module 'cudart.dll' (or one of its dependencies). Try using the full path with constructor syntax.
Finding libdevice from &lt;unknown&gt;
        searching for compute_20...     ERROR: can't open libdevice for compute_20
        searching for compute_30...     ERROR: can't open libdevice for compute_30
        searching for compute_35...     ERROR: can't open libdevice for compute_35
        searching for compute_50...     ERROR: can't open libdevice for compute_50


__ROC information__
ROC Available                                 : False
ROC Toolchains                                : None
HSA Agents Count                              : 0
HSA Agents:
None
HSA Discrete GPUs Count                       : 0
HSA Discrete GPUs                             : None

__SVML Information__
SVML State, config.USING_SVML                 : True
SVML Library Loaded                           : True
llvmlite Using SVML Patched LLVM              : True
SVML Operational                              : True

__Threading Layer Information__
TBB Threading Layer Available                 : False
+--&gt; Disabled due to Unknown import problem.
OpenMP Threading Layer Available              : True
+--&gt;Vendor: MS
Workqueue Threading Layer Available           : True
+--&gt;Workqueue imported successfully.

__Numba Environment Variable Information__
None found.

__Conda Information__
Conda Build                                   : 3.20.5
Conda Env                                     : 4.9.2
Conda Platform                                : win-64
Conda Python Version                          : 3.8.5.final.0
Conda Root Writable                           : True

__Installed Packages__
_pytorch_select           1.1.0                       cpu    anaconda
_tflow_select             2.3.0                       mkl    anaconda
absl-py                   0.12.0                   pypi_0    pypi
alabaster                 0.7.12                   pypi_0    pypi
appdirs                   1.4.3            py36h28b3542_0    anaconda
argparse                  1.4.0                    pypi_0    pypi
asn1crypto                1.3.0                    py36_0    anaconda
astor                     0.8.1              pyh9f0ad1d_0    conda-forge
astunparse                1.6.3                    pypi_0    pypi
atomicwrites              1.4.0                      py_0    anaconda
attrs                     19.3.0                     py_0    anaconda
babel                     2.9.0                    pypi_0    pypi
backcall                  0.2.0                      py_0    anaconda
backports                 1.0                        py_2    anaconda
backports.weakref         1.0.post1       py36h9f0ad1d_1001    conda-forge
blas                      1.0                         mkl    anaconda
bleach                    1.5.0                    py36_0    conda-forge
blinker                   1.4                        py_1    conda-forge
brotlipy                  0.7.0           py36he774522_1000    anaconda
bzip2                     1.0.8                he774522_0    anaconda
ca-certificates           2020.10.14                    0    anaconda
cachetools                4.1.1                      py_0    anaconda
certifi                   2020.6.20                py36_0    anaconda
cffi                      1.14.0           py36h7a1dbc1_0    anaconda
chardet                   3.0.4                 py36_1003    anaconda
click                     7.1.2              pyh9f0ad1d_0    conda-forge
cloudpickle               1.4.1                      py_0    anaconda
colorama                  0.4.3                      py_0    anaconda
contextlib2               0.6.0.post1                py_0    anaconda
cpuonly                   1.0                           0    pytorch
cryptography              2.9.2            py36h7a1dbc1_0    anaconda
cudatoolkit               9.0                           1    anaconda
cudnn                     7.6.5                 cuda9.0_0    anaconda
curl                      7.71.0               h2a8f88b_0    anaconda
cycler                    0.10.0           py36h009560c_0    anaconda
cython                    0.29.22                  pypi_0    pypi
cytoolz                   0.10.1           py36he774522_0    anaconda
dask-core                 2.19.0                     py_0    anaconda
decorator                 4.4.2                      py_0    anaconda
defusedxml                0.6.0                      py_0    anaconda
dlib                      19.20            py36h5653133_1    conda-forge
docker-py                 4.2.1            py36h9f0ad1d_0    conda-forge
docker-pycreds            0.4.0                      py_0    anaconda
docutils                  0.16                     pypi_0    pypi
easydict                  1.7                      pypi_0    pypi
entrypoints               0.3                      py36_0    anaconda
ffmpeg                    2.7.0                         0    menpo
flake8                    3.8.3                      py_0    anaconda
flake8-polyfill           1.0.2                    py36_0    anaconda
flake8-quotes             3.0.0              pyh9f0ad1d_0    conda-forge
flatbuffers               1.12                     pypi_0    pypi
freetype                  2.10.2               hd328e21_0    anaconda
gast                      0.2.2                    pypi_0    pypi
geos                      3.8.1                h33f27b4_0    anaconda
gettext                   0.19.8.1          hb01d8f6_1002    conda-forge
git                       2.23.0               h6bb4b03_0    anaconda
glib                      2.58.3          py36h04c7ab9_1004    conda-forge
google-auth               1.28.0                   pypi_0    pypi
google-auth-oauthlib      0.4.3                    pypi_0    pypi
google-pasta              0.2.0              pyh8c360ce_0    conda-forge
grpcio                    1.32.0                   pypi_0    pypi
h5py                      2.10.0           py36h5e291fa_0    anaconda
hdf5                      1.10.4               h7ebc959_0    anaconda
html5lib                  0.9999999                py36_0    conda-forge
icc_rt                    2019.0.0             h0cc432a_1    anaconda
icu                       58.2                 ha925a31_3    anaconda
idna                      2.10                       py_0    anaconda
imageio                   2.8.0                      py_0    anaconda
imageio-ffmpeg            0.4.2                      py_0    conda-forge
imagesize                 1.2.0                    pypi_0    pypi
imgaug                    0.4.0                    pypi_0    pypi
importlib-metadata        1.7.0                    py36_0    anaconda
importlib_metadata        1.7.0                         0    anaconda
intel-openmp              2019.4                      245    anaconda
ipykernel                 5.3.0            py36h5ca1d4c_0    anaconda
ipyparallel               6.3.0                    pypi_0    pypi
ipython                   7.16.1           py36h5ca1d4c_0    anaconda
ipython_genutils          0.2.0                    py36_0    anaconda
ipywidgets                7.5.1                      py_0    anaconda
jedi                      0.17.1                   py36_0    anaconda
jinja2                    2.11.2                     py_0    anaconda
joblib                    0.15.1                     py_0    anaconda
jpeg                      9d                   he774522_0    conda-forge
json-tricks               3.15.5                   pypi_0    pypi
jsonschema                3.2.0                    py36_0    anaconda
jupyter                   1.0.0                    py36_7    anaconda
jupyter_client            6.1.3                      py_0    anaconda
jupyter_console           6.1.0                      py_0    anaconda
jupyter_core              4.6.3                    py36_0    anaconda
keras-applications        1.0.8                      py_1    anaconda
keras-preprocessing       1.1.2                    pypi_0    pypi
kiwisolver                1.2.0            py36h74a9793_0    anaconda
krb5                      1.18.2               hc04afaa_0    anaconda
leptonica                 1.78.0               h919f142_2    conda-forge
libarchive                3.3.3                h0643e63_5    anaconda
libcurl                   7.71.0               h2a8f88b_0    anaconda
libffi                    3.2.1             h6538335_1007    conda-forge
libgpuarray               0.7.6             hfa6e2cd_1003    conda-forge
libiconv                  1.15             vc14h29686d3_5  [vc14]  anaconda
libmklml                  2019.0.5                      0    anaconda
libpng                    1.6.37               h2a8f88b_0    anaconda
libprotobuf               3.12.3               h7bd577a_0    anaconda
libsodium                 1.0.18               h62dcd97_0    anaconda
libssh2                   1.9.0                h7a1dbc1_1    anaconda
libtiff                   4.1.0                h56a325e_0    anaconda
libwebp                   1.0.2                hfa6e2cd_5    conda-forge
libxml2                   2.9.10               h464c3ec_1    anaconda
libxslt                   1.1.34               he774522_0    anaconda
lxml                      4.5.0            py36h1350720_0    anaconda
lz4-c                     1.8.1.2              h2fa13f4_0    anaconda
lzo                       2.10                 he774522_2    anaconda
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
mako                      1.1.0                      py_0    anaconda
markdown                  3.3.4                    pypi_0    pypi
markupsafe                1.1.1            py36he774522_0    anaconda
matplotlib                3.1.3                    py36_0    anaconda
matplotlib-base           3.1.3            py36h64f37c6_0    anaconda
mccabe                    0.6.1                    py36_1    anaconda
mistune                   0.8.4            py36he774522_0    anaconda
mkl                       2018.0.3                      1    anaconda
mkl_fft                   1.0.6            py36hdbbee80_0    anaconda
mkl_random                1.0.1            py36h77b88f5_1    anaconda
mock                      4.0.3                    pypi_0    pypi
more-itertools            8.4.0                      py_0    anaconda
moviepy                   1.0.1                      py_0    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
nbconvert                 5.6.1                    py36_0    anaconda
nbformat                  5.0.7                      py_0    anaconda
networkx                  2.4                        py_0    anaconda
ninja                     1.9.0            py36h74a9793_0    anaconda
nose                      1.3.7                    pypi_0    pypi
notebook                  6.0.3                    py36_0    anaconda
numpy                     1.19.5                   pypi_0    pypi
oauthlib                  3.1.0                      py_0    anaconda
olefile                   0.46                     py36_0    anaconda
opencv-python             3.4.1.15                 pypi_0    pypi
openjpeg                  2.3.1                h57dd2e7_3    conda-forge
openssl                   1.1.1h               he774522_0    anaconda
opt-einsum                3.3.0                    pypi_0    pypi
packaging                 20.4                       py_0    anaconda
pandas                    1.0.3            py36h47e9c7a_0    anaconda
pandoc                    2.9.2.1                       0    anaconda
pandocfilters             1.4.2                    py36_1    anaconda
parso                     0.7.0                      py_0    anaconda
pcre                      8.44                 ha925a31_0    anaconda
pep8-naming               0.8.2                    py36_0    anaconda
pickleshare               0.7.5                    py36_0    anaconda
pillow                    7.1.2            py36hcc1f983_0    anaconda
pip                       20.2.4                   py36_0    anaconda
pluggy                    0.13.1                   py36_0    anaconda
poppler                   0.87.0               hdbe765f_0    conda-forge
poppler-data              0.4.9                         1    conda-forge
proglog                   0.1.9                      py_0    conda-forge
prometheus_client         0.8.0                      py_0    anaconda
prompt-toolkit            3.0.5                      py_0    anaconda
prompt_toolkit            3.0.5                         0    anaconda
protobuf                  3.12.3           py36h33f27b4_0    anaconda
psutil                    5.8.0                    pypi_0    pypi
py                        1.9.0                      py_0    anaconda
pyasn1                    0.4.8                      py_0    anaconda
pyasn1-modules            0.2.8                    pypi_0    pypi
pycocotools               2.0                      pypi_0    pypi
pycodestyle               2.6.0                      py_0    anaconda
pycparser                 2.20                       py_0    anaconda
pyflakes                  2.2.0                      py_0    anaconda
pygments                  2.6.1                      py_0    anaconda
pygpu                     0.7.6           py36h7725771_1001    conda-forge
pyjwt                     1.7.1                      py_0    conda-forge
pyopenssl                 19.1.0                   py36_0    anaconda
pyparsing                 2.4.7                      py_0    anaconda
pyqt                      5.9.2            py36h6538335_2    anaconda
pyreadline                2.1                   py36_1001    conda-forge
pyrsistent                0.16.0           py36he774522_0    anaconda
pysocks                   1.7.1                    py36_0    anaconda
pytesseract               0.3.3              pyh8c360ce_0    conda-forge
pytest                    5.4.3                    py36_0    anaconda
python                    3.6.10               h9f7ef89_1    anaconda
python-dateutil           2.8.1                      py_0    anaconda
python_abi                3.6                     1_cp36m    conda-forge
pytorch                   1.5.1               py3.6_cpu_0  [cpuonly]  pytorch
pytz                      2020.1                     py_0    anaconda
pywavelets                1.1.1            py36he774522_0    anaconda
pywin32                   223              py36hfa6e2cd_1    anaconda
pywinpty                  0.5.7                    py36_0    anaconda
pyyaml                    5.3.1            py36he774522_0    anaconda
pyzmq                     19.0.1           py36ha925a31_1    anaconda
qt                        5.9.7            vc14h73c81de_0  [vc14]  anaconda
qtconsole                 4.7.5                      py_0    anaconda
qtpy                      1.9.0                      py_0    anaconda
requests                  2.24.0                     py_0    anaconda
requests-oauthlib         1.3.0              pyh9f0ad1d_0    conda-forge
rsa                       4.6                pyh9f0ad1d_0    conda-forge
scikit-image              0.16.2           py36h47e9c7a_0    anaconda
scikit-learn              0.20.1           py36hb854c30_0    anaconda
scipy                     1.4.1                    pypi_0    pypi
send2trash                1.5.0                    py36_0    anaconda
setuptools                50.3.0           py36h9490d1a_1    anaconda
shapely                   1.6.4                    pypi_0    pypi
simplejson                3.17.0           py36he774522_0    anaconda
sip                       4.19.8           py36h6538335_0    anaconda
six                       1.15.0                     py_0    anaconda
sklearn                   0.0                      pypi_0    pypi
slidingwindow             0.0.14                   pypi_0    pypi
snowballstemmer           2.1.0                    pypi_0    pypi
sphinx                    3.5.2                    pypi_0    pypi
sphinxcontrib-applehelp   1.0.2                    pypi_0    pypi
sphinxcontrib-devhelp     1.0.2                    pypi_0    pypi
sphinxcontrib-htmlhelp    1.0.3                    pypi_0    pypi
sphinxcontrib-jsmath      1.0.1                    pypi_0    pypi
sphinxcontrib-qthelp      1.0.3                    pypi_0    pypi
sphinxcontrib-serializinghtml 1.1.4                    pypi_0    pypi
sqlite                    3.32.3               h2a8f88b_0    anaconda
swig                      3.0.12               h047fa9f_3    anaconda
tbb                       2020.0               h74a9793_0    anaconda
tbb4py                    2020.0           py36h74a9793_0    anaconda
tensorboard               1.13.1                   pypi_0    pypi
tensorboard-plugin-wit    1.8.0                    pypi_0    pypi
tensorboardx              1.6                        py_0    conda-forge
tensorflow                2.4.1                    pypi_0    pypi
tensorflow-estimator      1.13.0                   pypi_0    pypi
tensorflow-gpu            1.13.1                   pypi_0    pypi
tensorflow-gpu-estimator  2.1.0                    pypi_0    pypi
termcolor                 1.1.0                    pypi_0    pypi
terminado                 0.8.3                    py36_0    anaconda
testpath                  0.4.4                      py_0    anaconda
theano                    1.0.4           py36h003fed8_1002    conda-forge
threadpoolctl             2.1.0              pyh5ca1d4c_0    anaconda
tk                        8.6.10               he774522_0    anaconda
toolz                     0.10.0                     py_0    anaconda
torchfile                 0.1.0                      py_0    conda-forge
torchvision               0.6.1                  py36_cpu  [cpuonly]  pytorch
tornado                   6.0.4            py36he774522_1    anaconda
tqdm                      4.47.0                     py_0    anaconda
traitlets                 4.3.3                    py36_0    anaconda
typing-extensions         3.7.4.3                  pypi_0    pypi
urllib3                   1.25.11                    py_0    anaconda
vc                        14.1                 h0510ff6_4    anaconda
visdom                    0.1.8.9                       0    conda-forge
vs2015_runtime            14.16.27012          hf0eaf9b_3    anaconda
vs2017_win-64             19.16.27038          h2e3bad8_2    conda-forge
vswhere                   2.7.1                h21ff451_0    anaconda
wcwidth                   0.2.5                      py_0    anaconda
webencodings              0.5.1                    py36_1    anaconda
websocket-client          0.57.0                   py36_1    anaconda
werkzeug                  1.0.1              pyh9f0ad1d_0    conda-forge
wget                      1.16.3                        0    menpo
wheel                     0.35.1                     py_0    anaconda
widgetsnbextension        3.5.1                    py36_0    anaconda
win_inet_pton             1.1.0                    py36_0    anaconda
wincertstore              0.2              py36h7fe50ca_0    anaconda
winpty                    0.4.3                         4    anaconda
wrapt                     1.12.1           py36h68a101e_1    conda-forge
xz                        5.2.5                h62dcd97_0    anaconda
yacs                      0.1.8                    pypi_0    pypi
yaml                      0.1.7                hc54c509_2    anaconda
zeromq                    4.3.2                ha925a31_2    anaconda
zipp                      3.3.1                      py_0    anaconda
zlib                      1.2.11               h62dcd97_4    anaconda
zstd                      1.3.7                h508b16e_0    anaconda

No errors reported.


Solved.
Pytorch was installing CPU only version for some reason, reinstalling pytorch didn't help.
Uninstalling pytorch: conda uninstall pytorch
Followed by uninstalling cpu only: conda uninstall cpuonly
Then installing pytorch again solved it.
