title: Using GPU inside docker container - CUDA Version: N/A and torch.cuda.is_available returns False
body: <p>I'm trying to use GPU from inside my docker container. I'm using docker with version 19.03 on Ubuntu 18.04.</p>
  <p>Outside the docker container if I run nvidia-smi I get the below output.</p>
  <pre><code>+-----------------------------------------------------------------------------+
  | NVIDIA-SMI 450.51.05    Driver Version: 450.51.05    CUDA Version: 11.0     |
  |-------------------------------+----------------------+----------------------+
  | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
  | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
  |                               |                      |               MIG M. |
  |===============================+======================+======================|
  |   0  Tesla T4            On   | 00000000:00:1E.0 Off |                    0 |
  | N/A   30C    P8     9W /  70W |      0MiB / 15109MiB |      0%      Default |
  |                               |                      |                  N/A |
  +-------------------------------+----------------------+----------------------+
  
  +-----------------------------------------------------------------------------+
  | Processes:                                                                  |
  |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
  |        ID   ID                                                   Usage      |
  |=============================================================================|
  |  No running processes found                                                 |
  +-----------------------------------------------------------------------------+
  </code></pre>
  <p>If I run the samething inside the container created from nvidia/cuda docker image, I get the same output as above and everything is running smoothly. torch.cuda.is_available() returns <strong>True</strong>.</p>
  <p>But If I run the same nvidia-smi command inside any other docker container, it gives the following output where you can see that the CUDA Version is coming as <strong>N/A</strong>. Inside the containers <strong>torch.cuda.is_available()</strong> also returns <strong>False</strong>.</p>
  <pre><code>+-----------------------------------------------------------------------------+
  | NVIDIA-SMI 450.51.05    Driver Version: 450.51.05    CUDA Version: N/A      |
  |-------------------------------+----------------------+----------------------+
  | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
  | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
  |                               |                      |               MIG M. |
  |===============================+======================+======================|
  |   0  Tesla T4            On   | 00000000:00:1E.0 Off |                    0 |
  | N/A   30C    P8     9W /  70W |      0MiB / 15109MiB |      0%      Default |
  |                               |                      |                  N/A |
  +-------------------------------+----------------------+----------------------+
  
  +-----------------------------------------------------------------------------+
  | Processes:                                                                  |
  |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
  |        ID   ID                                                   Usage      |
  |=============================================================================|
  |  No running processes found                                                 |
  +-----------------------------------------------------------------------------+
  </code></pre>
  <p>I have installed nvidia-container-toolkit using the following commands.</p>
  <pre><code>curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
  curl -s -L https://nvidia.github.io/nvidia-docker/ubuntu18.04/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
  sudo apt-get update
  sudo apt-get install nvidia-container-toolkit
  sudo systemctl restart docker
  </code></pre>
  <p>I started my containers using the following commands</p>
  <pre><code>sudo docker run --rm --gpus all nvidia/cuda nvidia-smi
  sudo docker run -it --rm --gpus all ubuntu nvidia-smi
  </code></pre>