title: Tensorflow inference very slow on first time run in docker
body: <p>I built a docker image for a server that does inference with tensorflow. I installed tensorflow-gpu with pip in the docker image. It works fine for my machine with titan x gpus. But when I ran the docker container on another machine with 1080 ti gpus. The first run becomes incredibly slow, takes about 90 seconds, usually it takes 7 seconds on the first run and 1 second in the following runs. I tried to set tf_cudnn_use_autotune to 0, and also mount a folder to save the cuda cache. But it doesn't really solve the problem. Any one has any suggestion?</p>