Could not find any nvml.h during building from source in docker 
@YuMJie,
The latest tensorflow stable v2.16.1 will be compatible with Clang 17.0.6, Bazel 6.5.0, cuDNN8.9 and CUDA 12.3. COuld you please try to follow the same while installing. The nvml.h header file is part of the NVIDIA Management Library (NVML)  which allows programmatic interaction with NVIDIA GPUs.  NVML typically comes bundled with the NVIDIA display driver. Check if you have the latest drivers installed from the official NVIDIA website.

https://github.com/openxla/xla/issues/6923

Thank you! I found why did not find the nvml.h: If I run `git pull`  in the dir tensorflow_src  and run building ,it will occur this error. However ,  directly running the build will not have this error . Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/65999">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/65999">No</a>
 If you don't run `git pull` then it is building a version from 2022 and not a more recent version. The part of the NVIDIA driver does not seem to be installed but that could be because the container assumes you have the driver on the host system, not in the container?
### Issue type

Build/Install

### Have you reproduced the bug with TensorFlow Nightly?

Yes

### Source

source

### TensorFlow version

2.16.1

### Custom code

Yes

### OS platform and distribution

linux ubutun 20.04

### Mobile device

linux ubutun 20.04

### Python version

3.11

### Bazel version

7.1.1

### GCC/compiler version

11.4.0

### CUDA/cuDNN version

12.3

### GPU model and memory

3090 24G

### Current behavior?

I try to build the tensorflow from source code. So I download the docker image and build. However it error during the `run ./configure`


### Standalone code to reproduce the issue

```shell
$ git clone https://github.com/tensorflow/tensorflow.git 
$ docker pull tensorflow/tensorflow:lastest-gpu
$ docker run --gpus all -it -w /tensorflow -v $PWD:/mnt -e HOST_PERMS="$(id -u):$(id -g)" \
    tensorflow/tensorflow:lastest-gpu bash

# within the container
$ cd mnt/tensorflow
$ ./configure
```


### Relevant log output

```shell

root@27cd49d821ca:/mnt/tensorflow# nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Wed_Nov_22_10:17:15_PST_2023
Cuda compilation tools, release 12.3, V12.3.107
Build cuda_12.3.r12.3/compiler.33567101_0



root@27cd49d821ca:/mnt/tensorflow# ./configure 
WARNING: current bazel installation is not a release version.
Please specify the location of python. [Default is /usr/bin/python3]: 


Found possible Python library paths:
  /usr/lib/python3/dist-packages
  /usr/local/lib/python3.11/dist-packages
Please input the desired Python library path to use.  Default is [/usr/lib/python3/dist-packages]
/usr/local/lib/python3.11/dist-packages
Do you wish to build TensorFlow with ROCm support? [y/N]: N
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: Y
CUDA support will be enabled for TensorFlow.

Do you wish to build TensorFlow with TensorRT support? [y/N]: N
No TensorRT support will be enabled for TensorFlow.

Could not find any nvml.h in any subdirectory:
        ''
        'include'
        'include/cuda'
        'include/*-linux-gnu'
        'extras/CUPTI/include'
        'include/cuda/CUPTI'
        'local/cuda/extras/CUPTI/include'
        'targets/x86_64-linux/include'
of:
        '/lib/x86_64-linux-gnu'
        '/usr'
        '/usr/local/cuda'
        '/usr/local/cuda/lib64/stubs'
        '/usr/local/cuda/targets/x86_64-linux/lib'

Asking for detailed CUDA configuration...

Please specify the CUDA SDK version you want to use. [Leave empty to default to CUDA 11]: 12.3


Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 2]: 


Please specify the locally installed NCCL version you want to use. [Leave empty to use http://github.com/nvidia/nccl]: 


Please specify the comma-separated list of base paths to look for CUDA libraries and headers. [Leave empty to use the default]: 


Could not find any nvml.h in any subdirectory:
        ''
        'include'
        'include/cuda'
        'include/*-linux-gnu'
        'extras/CUPTI/include'
        'include/cuda/CUPTI'
        'local/cuda/extras/CUPTI/include'
        'targets/x86_64-linux/include'
of:
        '/lib/x86_64-linux-gnu'
        '/usr'
        '/usr/local/cuda'
        '/usr/local/cuda-12.3'
        '/usr/local/cuda/lib64/stubs'
        '/usr/local/cuda/targets/x86_64-linux/lib'

Asking for detailed CUDA configuration...

Please specify the CUDA SDK version you want to use. [Leave empty to default to CUDA 11]:
```
