Manual install vs conda install tensorflow-gpu
Apparently all Cuda and CuDnn libraries are included with packages from Anaconda Cloud according to this guy:  https://www.pugetsystems.com/labs/hpc/How-To-Install-CUDA-10-together-with-9-2-on-Ubuntu-18-04-with-support-for-NVIDIA-20XX-Turing-GPUs-1236/


  You do NOT need to install CUDA 10 if; You are a developer working
  with CUDA 9.x or earlier in your code base and there are no compelling
  new features that you want to risk breaking your application to try.
  See the release notes. You are working with Python machine learning
  frameworks AND using Anaconda Python. Most of the frameworks have
  up-to-date builds available from Anaconda Cloud that include the
  needed CUDA libraries i.e. you don't have to install anything outside
  of the package you want. You are using GPU accelerated software
  packages that only depend on CUDA run-time support from the display
  driver. That includes most "ready to use" or commercial software. You
  don't know what CUDA is. In that case then, well, thanks for reading
  my post. I hope you enjoy it and learn something from it.


Check out this advice:  https://towardsdatascience.com/tensorflow-gpu-installation-made-easy-use-conda-instead-of-pip-52e5249374bc

He says just create a conda env like conda create --name tf_gpu tensorflow-gpu where tf_gpu is the env name and tensorflow-gpu is to include cuda and cudnn in that env.  Breaking it down into separate commands, it looks like:

conda create --name tf_gpu
activate tf_gpu
conda install tensorflow-gpu