How does one install pytorch 1.9 in an HPC that seems to refuse to cooperate?
I've been trying to install PyTorch 1.9 with Cuda (ideally 11) on my HPC but I cannot.
The cluster says:
Package typing-extensions conflicts for:
typing-extensions
torchvision -&gt; pytorch==1.8.1 -&gt; typing-extensionsThe following specifications were found to be incompatible with your system:

  - feature:/linux-64::__glibc==2.17=0
  - feature:|@/linux-64::__glibc==2.17=0
  - cffi -&gt; libgcc-ng[version='&gt;=7.5.0'] -&gt; __glibc[version='&gt;=2.17']
  - cudatoolkit=11.0 -&gt; __glibc[version='&gt;=2.17,&lt;3.0.a0']
  - cudatoolkit=11.0 -&gt; libgcc-ng[version='&gt;=7.3.0'] -&gt; __glibc[version='&gt;=2.17']
  - freetype -&gt; libgcc-ng[version='&gt;=7.3.0'] -&gt; __glibc[version='&gt;=2.17']
  - jpeg -&gt; libgcc-ng[version='&gt;=7.5.0'] -&gt; __glibc[version='&gt;=2.17']
  - lcms2 -&gt; libgcc-ng[version='&gt;=7.3.0'] -&gt; __glibc[version='&gt;=2.17']
  - libffi -&gt; libgcc-ng[version='&gt;=7.3.0'] -&gt; __glibc[version='&gt;=2.17']
  - libgcc-ng -&gt; __glibc[version='&gt;=2.17']
  - libmklml -&gt; libgcc-ng -&gt; __glibc[version='&gt;=2.17']
  - libpng -&gt; libgcc-ng[version='&gt;=7.3.0'] -&gt; __glibc[version='&gt;=2.17']
  - libstdcxx-ng -&gt; __glibc[version='&gt;=2.17']
  - libtiff -&gt; libgcc-ng[version='&gt;=7.3.0'] -&gt; __glibc[version='&gt;=2.17']
  - libwebp-base -&gt; libgcc-ng[version='&gt;=7.3.0'] -&gt; __glibc[version='&gt;=2.17']
  - lz4-c -&gt; libgcc-ng[version='&gt;=7.5.0'] -&gt; __glibc[version='&gt;=2.17']
  - mkl-service -&gt; libgcc-ng[version='&gt;=7.5.0'] -&gt; __glibc[version='&gt;=2.17']
  - mkl_fft -&gt; libgcc-ng[version='&gt;=7.3.0'] -&gt; __glibc[version='&gt;=2.17']
  - mkl_random -&gt; libgcc-ng[version='&gt;=7.5.0'] -&gt; __glibc[version='&gt;=2.17']
  - ncurses -&gt; libgcc-ng[version='&gt;=7.3.0'] -&gt; __glibc[version='&gt;=2.17']
  - ninja -&gt; libgcc-ng[version='&gt;=7.3.0'] -&gt; __glibc[version='&gt;=2.17']
  - numpy -&gt; libgcc-ng[version='&gt;=7.5.0'] -&gt; __glibc[version='&gt;=2.17']
  - numpy-base -&gt; libgcc-ng[version='&gt;=7.5.0'] -&gt; __glibc[version='&gt;=2.17']
  - openjpeg -&gt; libgcc-ng[version='&gt;=7.5.0'] -&gt; __glibc[version='&gt;=2.17']
  - openssl -&gt; libgcc-ng[version='&gt;=7.5.0'] -&gt; __glibc[version='&gt;=2.17']
  - pillow -&gt; libgcc-ng[version='&gt;=7.5.0'] -&gt; __glibc[version='&gt;=2.17']
  - python=3.9 -&gt; libgcc-ng[version='&gt;=7.5.0'] -&gt; __glibc[version='&gt;=2.17']
  - pytorch==1.9 -&gt; cudatoolkit[version='&gt;=11.1,&lt;11.2'] -&gt; __glibc[version='&gt;=2.17,&lt;3.0.a0']
  - readline -&gt; libgcc-ng[version='&gt;=7.3.0'] -&gt; __glibc[version='&gt;=2.17']
  - sqlite -&gt; libgcc-ng[version='&gt;=7.5.0'] -&gt; __glibc[version='&gt;=2.17']
  - tk -&gt; libgcc-ng[version='&gt;=7.3.0'] -&gt; __glibc[version='&gt;=2.17']
  - torchvision -&gt; cudatoolkit[version='&gt;=11.1,&lt;11.2'] -&gt; __glibc[version='&gt;=2.17|&gt;=2.17,&lt;3.0.a0']
  - xz -&gt; libgcc-ng[version='&gt;=7.3.0'] -&gt; __glibc[version='&gt;=2.17']
  - zlib -&gt; libgcc-ng[version='&gt;=7.3.0'] -&gt; __glibc[version='&gt;=2.17']
  - zstd -&gt; libgcc-ng[version='&gt;=7.3.0'] -&gt; __glibc[version='&gt;=2.17']

Your installed version is: 2.17

but I don't understand how to use that info to install it. Is it something I can do for the system admins?

When I try to install it with conda, I get a message telling me that it's already installed. However, a conda list greps shows the version is only CPU, not GPU:
(metalearning_gpu) miranda9~/automl-meta-learning $ conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia

Collecting package metadata (current_repodata.json): done
Solving environment: done

# All requested packages already installed.

(metalearning_gpu) miranda9~/automl-meta-learning $ 
(metalearning_gpu) miranda9~/automl-meta-learning $ conda list | grep torch
cpuonly                   1.0                           0    pytorch
ffmpeg                    4.3                  hf484d3e_0    pytorch
pytorch                   1.9.0               py3.9_cpu_0  [cpuonly]  pytorch
torch                     1.9.0+cpu                pypi_0    pypi
torchaudio                0.9.0                    pypi_0    pypi
torchmeta                 1.7.0                    pypi_0    pypi
torchvision               0.10.0+cpu               pypi_0    pypi

Attempting to install it with pip completely fails:
(metalearning_gpu) miranda9~/automl-meta-learning $ pip3 install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html


Looking in links: https://download.pytorch.org/whl/torch_stable.html
Collecting torch==1.9.0+cu111
ERROR: Exception:
Traceback (most recent call last):
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/cli/base_command.py&quot;, line 173, in _main
    status = self.run(options, args)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/cli/req_command.py&quot;, line 203, in wrapper
    return func(self, options, args)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/commands/install.py&quot;, line 315, in run
    requirement_set = resolver.resolve(
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py&quot;, line 94, in resolve
    result = self._result = resolver.resolve(
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py&quot;, line 472, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py&quot;, line 341, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py&quot;, line 172, in _add_to_criteria
    if not criterion.candidates:
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/resolvelib/structs.py&quot;, line 151, in __bool__
    return bool(self._sequence)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py&quot;, line 140, in __bool__
    return any(self)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py&quot;, line 128, in &lt;genexpr&gt;
    return (c for c in iterator if id(c) not in self._incompatible_ids)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py&quot;, line 32, in _iter_built
    candidate = func()
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py&quot;, line 204, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py&quot;, line 295, in __init__
    super().__init__(
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py&quot;, line 156, in __init__
    self.dist = self._prepare()
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py&quot;, line 227, in _prepare
    dist = self._prepare_distribution()
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py&quot;, line 305, in _prepare_distribution
    return self._factory.preparer.prepare_linked_requirement(
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/operations/prepare.py&quot;, line 508, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/operations/prepare.py&quot;, line 550, in _prepare_linked_requirement
    local_file = unpack_url(
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/operations/prepare.py&quot;, line 239, in unpack_url
    file = get_http_url(
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/operations/prepare.py&quot;, line 102, in get_http_url
    from_path, content_type = download(link, temp_dir.path)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/network/download.py&quot;, line 132, in __call__
    resp = _http_get_download(self._session, link)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/network/download.py&quot;, line 115, in _http_get_download
    resp = session.get(target_url, headers=HEADERS, stream=True)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/requests/sessions.py&quot;, line 555, in get
    return self.request('GET', url, **kwargs)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/network/session.py&quot;, line 454, in request
    return super().request(method, url, *args, **kwargs)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/requests/sessions.py&quot;, line 542, in request
    resp = self.send(prep, **send_kwargs)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/requests/sessions.py&quot;, line 655, in send
    r = adapter.send(request, **kwargs)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/cachecontrol/adapter.py&quot;, line 44, in send
    cached_response = self.controller.cached_request(request)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/cachecontrol/controller.py&quot;, line 139, in cached_request
    cache_data = self.cache.get(cache_url)
  File &quot;/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/network/cache.py&quot;, line 54, in get
    return f.read()
MemoryError

Current install script:
## Installation script
# to install do: bash ~/automl-meta-learning/install.sh

#conda update conda

#conda create -y -n metalearning_gpu python=3.9
#conda activate metalearning_gpu
#conda remove --name metalearning_gpu --all

module load cuda-toolkit/11.1
module load gcc/9.2.0

# A40, needs cuda at least 11.0, but 1.9 requires 11
conda activate metalearning_gpu
conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia
pip3 install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html

#conda activate metalearning_cpu
#conda install pytorch torchvision torchaudio cpuonly -c pytorch
#pip3 install torch==1.9.0+cpu torchvision==0.10.0+cpu torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html

# uutils installs
conda install -y dill
conda install -y networkx&gt;=2.5
conda install -y scipy
conda install -y scikit-learn
conda install -y lark-parser -c conda-forge

# due to compatibility with torch=1.7.x, https://stackoverflow.com/questions/65575871/torchtext-importerror-in-colab
#conda install -y torchtext==0.8.0 -c pytorch

conda install -y tensorboard
conda install -y pandas
conda install -y progressbar2
conda install -y transformers
conda install -y requests
conda install -y aiohttp
conda install -y numpy
conda install -y plotly
conda install -y matplotlib

pip install wandb

# for automl
conda install -y pyyml
conda install -y torchviz
#conda install -y graphviz

#pip install tensorflow
#pip install learn2learn

#pip install -U git+https://github.com/brando90/pytorch-meta.git
#pip install --no-deps torchmeta==1.6.1
pip install --no-deps torchmeta==1.7.0
#        'torch&gt;=1.4.0,&lt;1.9.0',
#        'torchvision&gt;=0.5.0,&lt;0.10.0',
#pip install -y numpy
pip install Pillow
pip install h5py
#pip install requests
pip install ordered-set

pip install higher
#    'torch'

#pip install -U git+https://github.com/moskomule/anatome
pip install --no-deps -U git+https://github.com/moskomule/anatome
#    'torch&gt;=1.9.0',
#    'torchvision&gt;=0.10.0',
pip install tqdm

# - using conda develop rather than pip because uutils installs incompatible versions with the vision cluster
## python -c &quot;import sys; [print(p) for p in sys.path]&quot;
conda install conda-build
# conda develop ~/ultimate-utils/ultimate-utils-proj-src
# conda develop ~/automl-meta-learning/automl-proj-src
# pip install ultimate-utils



# -- extra notes

# local editable installs
# HAL installs, make sure to clone from wmlce 1.7.0 that has h5py ~= 2.9.0 and torch 1.3.1 and torchvision 0.4.2
# pip install torchmeta==1.3.1


First of all, as @Francois suggested, try to uninstall the CPU only version of pytorch. Also in your installation script, you should use either conda or pip3.
Then you may want to try the following attempts:

using conda: add conda-forge channel  to your command (conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia -c conda-forge). And make sure conda is updated.
using pip: insert --no-cache-dir into your command (pip3 --no-cache-dir install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html) to avoid the MemoryError.

