"CUDA_ERROR_NOT_FOUND: named symbol not found" in Docker container
I have the same issue: #68710 Please make sure that you have installed the Nvidia container toolkit before. Your docker container is not able to find the relevant CUDA, hence the error. Here's the documention: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html I have tried that already and it did not make a difference. AFAIK it's only necessary when running Docker on a native Linux host. In my configuration with WSL2 it should already be covered by Docker Desktop.
I have found this YouTube tutorial where it works without the container Toolkit too: https://youtu.be/YozfiLI1ogY. **I'm getting the same issue, within the docker image  tensorflow/tensorflow:latest-gpu-jupyter (2.16.1) I am getting...

# ./__nvcc_device_query
./__nvcc_device_query failed to call cudaLoader::cuInit(0) with error 0x1f4 (CUDA_ERROR_NOT_FOUND)
# ./nvcc --version 
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Wed_Nov_22_10:17:15_PST_2023
Cuda compilation tools, release 12.3, V12.3.107
Build cuda_12.3.r12.3/compiler.33567101_0
# nvidia-smi
Tue May 28 10:03:54 2024       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 555.42.03              Driver Version: 555.85         CUDA Version: 12.5     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3070        On  |   00000000:02:00.0  On |                  N/A |
|  0%   48C    P8             21W /  220W |     637MiB /   8192MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+** I'm getting the same issue with tensorflow/tensorflow:2.13.0-gpu-jupyter ....
E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:268] failed call to cuInit: CUDA_ERROR_NOT_FOUND: named symbol not found I can install and run CUDA samples in WSL....(Ubuntu 22.04) by installing the CUDA tool kit 12.5

Device 0: "NVIDIA GeForce RTX 3070" with Compute 8.6 capability
printf() is called. Output:

[2, 0]:         Value is:10
[2, 1]:         Value is:10
[2, 2]:         Value is:10
[2, 3]:         Value is:10
[2, 4]:         Value is:10
[2, 5]:         Value is:10
[2, 6]:         Value is:10
[2, 7]:         Value is:10
[3, 0]:         Value is:10 I have found a solution and shared it here: #68710  The workaround is working. Since this issue is pretty much a duplicate of the other one I will close it now. The other one should be left open until the latest drivers is working again. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/68711">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/68711">No</a>
 

Nvidia 555.85 not working. Cuda 12.5 buggy

Use 551.86 game ready driver and Cuda 12.4 with docker tensorflow/tensorflow:latest-gpu

### Issue type

Bug

### Have you reproduced the bug with TensorFlow Nightly?

Yes

### Source

binary

### TensorFlow version

2.16.1-0-g5bc9d26649c

### Custom code

No

### OS platform and distribution

Windows 11 Pro + WSL2 Ubuntu 22.04

### Mobile device

_No response_

### Python version

3.11.0rc1

### Bazel version

_No response_

### GCC/compiler version

_No response_

### CUDA/cuDNN version

_No response_

### GPU model and memory

RTX 4080

### Current behavior?

I am using Docker Desktop with WSL2 on Windows 11 Pro to run the `tensorflow-gpu:latest` Docker container. After starting it no GPU is found and the error `failed call to cuInit: CUDA_ERROR_NOT_FOUND: named symbol not found` occurs. CUDA, cuDNN and Python versions are all dictated by the Docker image so I cannot change them.

Things I have tried without success:
- using a different container version (2.16.1, 2.15.0, 2.14.0, 2.13.0)
- reinstalling WSL2/Ubuntu
- reinstalling Docker Desktop
- reinstalling GPU drivers

Reinstalling the drivers is the only thing that had any effect at all. It changed the error from CUDA_ERROR_NO_DEVICE to CUDA_ERROR_NOT_FOUND.

I have found https://github.com/google/jax/issues/13570 with a similar error but it's fix doesn't apply here. RTX 4000 cards require CUDA >=11.8 which I am using.

### Standalone code to reproduce the issue

```shell
docker run --rm --gpus all -it tensorflow/tensorflow:latest-gpu bash
python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"
```


### Relevant log output

```shell
2024-05-27 18:05:30.149964: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-27 18:05:31.089452: E external/local_xla/xla/stream_executor/cuda/cuda_driver.cc:282] failed call to cuInit: CUDA_ERROR_NOT_FOUND: named symbol not found
[]
```
