TF-TRT Warning: Could not find TensorRT
Hi @mikechen66 ,

Please follow the correct configuration and follow the [documentation](https://www.tensorflow.org/install/pip) 

Please also have a look of similar issue at https://github.com/tensorflow/tensorflow/issues/41060. 

Thanks! This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. does anyone have any updates on this? facing the same issue without a trace in sight Hi @mikechen66 ,

I am assuming you are using prebuilt binaries from Pypi.If you are building from source please confirm.

Could you please confirm whether CUDNN PATH setting done like below. Refer step 4 in attached [documentation](https://www.tensorflow.org/install/pip#step-by-step_instructions) source for more details on configuring GPU.

```
CUDNN_PATH=$(dirname $(python -c "import nvidia.cudnn;print(nvidia.cudnn.__file__)"))
export LD_LIBRARY_PATH=$CUDNN_PATH/lib:$CONDA_PREFIX/lib/:$LD_LIBRARY_PATH
```

Also please run the command `nvidia-smi` to check whether Nvidia driver installed or not.

Please ignore TensorRT warning as this is optional and it won't affect GPU. Thanks! This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. still having issues too I have the same issue. Does anyone have any updates on this?  I too am suffering from this issue. Try adding this command

`!pip install tensorflow-gpu==2.8.0` > Try adding this command
> 
> `!pip install tensorflow-gpu==2.8.0`

Thank you, it works for me. > Try adding this command
> 
> `!pip install tensorflow-gpu==2.8.0`

It seems tensorflow-gpu will be deprecated? #60074 Looks like it, but it works for me as well. Starting from TF2.12v `tensorflow-gpu` became redundant and installing `tensorflow` automatically installs GPU version also. @mikechen66 , Can we mark it as closed. Please let us know if still having concerns.

Thanks! had same issue as others

> ```
> CUDNN_PATH=$(dirname $(python -c "import nvidia.cudnn;print(nvidia.cudnn.__file__)"))
> export LD_LIBRARY_PATH=$CUDNN_PATH/lib:<path to your python libs>/tensorrt_libs/:$LD_LIBRARY_PATH:
> ```
from @SuryanarayanaY & [SO](https://stackoverflow.com/a/76882097) seemed to do the trick for me
in case others had the same issue but didn't install tensorflow through conda

Note: will need to add to environment variables in profile or `.env` file otherwise will need to reset these variables before each run > had same issue as others
> 
> > ```
> > CUDNN_PATH=$(dirname $(python -c "import nvidia.cudnn;print(nvidia.cudnn.__file__)"))
> > export LD_LIBRARY_PATH=$CUDNN_PATH/lib:<path to your python libs>/tensorrt_libs/:$LD_LIBRARY_PATH:
> > ```
> 
> from @SuryanarayanaY & [SO](https://stackoverflow.com/a/76882097) seemed to do the trick for me in case others had the same issue but didn't install tensorflow through conda
> 
> Note: will need to add to environment variables in profile or `.env` file otherwise will need to reset these variables before each run

This works for me! Thanks. This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. > had same issue as others
> 
> > ```
> > CUDNN_PATH=$(dirname $(python -c "import nvidia.cudnn;print(nvidia.cudnn.__file__)"))
> > export LD_LIBRARY_PATH=$CUDNN_PATH/lib:<path to your python libs>/tensorrt_libs/:$LD_LIBRARY_PATH:
> > ```
> 
> from @SuryanarayanaY & [SO](https://stackoverflow.com/a/76882097) seemed to do the trick for me in case others had the same issue but didn't install tensorflow through conda
> 
> Note: will need to add to environment variables in profile or `.env` file otherwise will need to reset these variables before each run

Could you tell the command to find path to python libs ?
Also I am unable to find tensor_libs folder using find command. In my case the log reported 
```
TF-TRT Warning: Could not find TensorRT
```
but my program ran otherwise normally. I resolved this by using TensorRT 8 instead of TensorRT 7, contrary to what the [documentation](https://www.tensorflow.org/install/pip#software_requirements) suggests.

After tracking the source code a bit, it seems that library search starts in [`py_utils.cc`](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/compiler/tf2tensorrt/utils/py_utils.cc#L36) and lands in [`load_library.cc`](https://github.com/google/tsl/blob/main/tsl/platform/default/load_library.cc#L27). So we can use `dlopen` to test what gets wrong (under Linux, for example).

I used the following snippet to check the actual error when loading the TensorRT library:
```c
#include <dlfcn.h>
#include <stdio.h>

int main() {
  void* handle = dlopen("libnvinfer.so", RTLD_NOW | RTLD_LOCAL);
  if (handle) {
    printf("%p", handle);
    dlclose(handle);
  } else {
    printf("%s", dlerror());
  }
  printf("\n");
  return handle == NULL;
}
```
In my case, it printed
```
libnvrtc.so.11.1: cannot open shared object file: No such file or directory
```
and I could only find `libnvrtc.so.11.2` in my system, which seems too new for TensorRT 7. After being aware that [TensorFlow supports TensorRT 8](https://github.com/tensorflow/tensorflow/pull/52932), I upgraded TensorRT and the warning was gone. Hi @mikechen66 ,

This could not be an issue with latest version TF2.14v. As tensorflow team came up with GPU package that also bundled with all the necessary cuda,cudnn and Tensorrt packages etc. You can use `pip install tensorflow[and-cuda]==2.14` and it installs all the comapatable versions of GPU packages also. This will ensure there is no compatibility issues arises in any of cuda,cudnn, tensorrt etc.

https://github.com/tensorflow/tensorflow/blob/4dacf3f368eb7965e9b5c3bbdd5193986081c3b2/tensorflow/tools/pip_package/setup.py#L169-L181

Please verify and confirm if there is still an issue. Thanks! This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. > Try adding this command
> 
> `!pip install tensorflow-gpu==2.8.0`

it's worked thanks The following snippet resolved the tensorrt problem completely on my machine.
```
mkdir -p $CONDA_PREFIX/etc/conda/activate.d

echo 'CUDNN_PATH=$(dirname $(python -c "import nvidia.cudnn;print(nvidia.cudnn.__file__)"))' >> $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh
echo 'TENSORRT_PATH=$(dirname $(python -c "import tensorrt;print(tensorrt.__file__)"))' >> $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh
echo 'export LD_LIBRARY_PATH=$CONDA_PREFIX/lib/:$CUDNN_PATH/lib:$LD_LIBRARY_PATH' >> $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh

source $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh
``` This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61468">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61468">No</a>
 FWIW, similarly to the answers above setting the LD_LIBRARY_PATH path resolved the issue on my machine (I am not using conda). I downloaded `TensorRT-8.6.1.6.Linux.x86_64-gnu.cuda-12.0` and `cudnn-linux-x86_64-8.9.7.29_cuda12-archive` from Nvidia for my version of tensorrt (8.6.1.post1) . You need to have both as otherwise the msg will be printed and I suspect `TryDlopenTensorRTLibraries` requires a number of libs to be there which `pip install tensorrt` does not fetch :thinking: . @skonto you're right, 2.15-post1 is looking for `libnvinfer_plugin.so.8.6.1` and libnvinfer.so.8.6.1

```c++
absl::StatusOr<void*> GetNvInferDsoHandle() {
#if defined(PLATFORM_WINDOWS)
  return GetDsoHandle("nvinfer", "");
#else
  return GetDsoHandle("nvinfer", GetTensorRTVersion());
#endif
}

absl::StatusOr<void*> GetNvInferPluginDsoHandle() {
#if defined(PLATFORM_WINDOWS)
  return GetDsoHandle("nvinfer_plugin", "");
#else
  return GetDsoHandle("nvinfer_plugin", GetTensorRTVersion());
#endif
}
```

You can know which file your tf is finding by using `strace -e open,openat python -c "import tensorflow as tf"` in your venv.

I recall that in before, tf2.10 and below looks for libnvinfer.so.7 while the tensorrt python prebuild package only offers libnvinfer.so.8, so you do stupid things `ln -s libnvinfer.so.8 libnvinfer.so.7` to get it working,

also the file used to be in tensorrt folder, but now, the tensorrt package is not included in `pip install tensorflow[and-cuda]` (and-cuda package). Thus you need to install tensorrt, too. The tensorrt package on pypi comes with 8.6.1. However, the package only gives libnvinfer.so.8 where tensorflow is looking for libnvinfer.8.6.1.

Thus the way to solve this is to go to your venv site-packages folder, find tensorrt_libs folder,

(in my case tf version 2.15-post1)

`ln -s libnvinfer_plugin.so.8 libnvinfer_plugin.so.8.6.1`

`ln -s libnvinfer.so.8 libnvinfer.so.8.6.1`

and make sure tensorrt_libs folder is in the LD_LIBRARY_PATH

(maybe `TensorRT-8.6.1.6.Linux.x86_64-gnu.cuda-12.0` offers `libnvinfer.so.8.6.1`, but it works for me though)

 Where do I need to copy these 2 files to?
for ex. TensorRT-8.6.1.6.Linux.x86_64-gnu.cuda-12.0
fyii. I am on Ubuntu 22.04 on WSL2 on Windows 11 It's quite difficult to manage everything with all those changes during time, but a logical assumption is that `tensorflow[and-cuda]` will remain to be a preferable setup in the future. So, for the most recent tensorflow and CUDA:

`pip install tensorflow[and-cuda]==v2.16.0-rc0`

```
python3 -c "import tensorflow.compiler as tf_cc; \
print(tf_cc.tf2tensorrt._pywrap_py_utils.get_linked_tensorrt_version())"
```
(8, 6, 1)

Download related tar.gz https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/secure/8.6.1/tars/TensorRT-8.6.1.6.Linux.x86_64-gnu.cuda-12.0.tar.gz , extract it and set its `lib` to the environment variable LD_LIBRARY_PATH:

```
export CUDNN_PATH=$HOME/venv/lib/python3.11/site-packages/nvidia/cudnn
export LD_LIBRARY_PATH=$CUDNN_PATH/lib:$HOME/repos/TensorRT-8.6.1.6/lib:$LD_LIBRARY_PATH
export TF_ENABLE_ONEDNN_OPTS=0
```
 for me, debian 12, in docker environment, i had to install ` pip install tensorflow[and-cuda]==2.15.1`  so it list the gpu for version 2.16.x, it does not list anymore
### Issue type

Build/Install

### Have you reproduced the bug with TensorFlow Nightly?

Yes

### Source

source

### TensorFlow version

tf2.12, tf2.13

### Custom code

Yes

### OS platform and distribution

Linux Ubuntu 20.04

### Mobile device

_No response_

### Python version

3.10, 3.11

### Bazel version

_No response_

### GCC/compiler version

9.40

### CUDA/cuDNN version

CUDA 11.8, cuDNN8.6

### GPU model and memory

RTX2060

### Current behavior?

import tensorflow as tf2023-08-03 17:42:07.337886: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-08-03 17:42:07.926267: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT


### Standalone code to reproduce the issue

```shell
$ conda create --name tf python=3.10
$ conda activate tf

$ conda install -c conda-forge cudatoolkit=11.8.0
$ pip install nvidia-cudnn-cu11==8.6.0.163


mkdir -p $CONDA_PREFIX/etc/conda/activate.d
echo 'CUDNN_PATH=$(dirname $(python -c "import nvidia.cudnn;print(nvidia.cudnn.__file__)"))' >> $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh
echo 'export LD_LIBRARY_PATH=$CONDA_PREFIX/lib/:$CUDNN_PATH/lib:$LD_LIBRARY_PATH' >> $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh
source $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh

import tensorflow as tf
print(tf.config.list_physical_devices('GPU'))
```


### Relevant log output

```shell
I have already installed cudatoolkit 11.8 written in the above commands. But It shows CUDA Toolkit is not installed while giving the command. 

$ nvcc --version
CUDA Toolkit is not installed.
```
