title: Performing inference in Tensorflow with multiple graphs on same GPU parallely
body: <p>I created 2 tf.graph() (loaded the same frozen model) from different threads, which seem to happen based on the different tf.graph object address that I see in each thread. When I tried to run inference on these 2 graphs parallely using the same GPU, I feel it is running like a sequential pipeline.</p>
  
  <p>This is based on the GPU memory &amp; utilization stats and also the time taken. ie, A single graph would consume 7GB of GPU memory and utilize ~15% of compute. With 2 graphs running parallely I would expect to use twice the GPU memory and increased compute, but that doesn't seem to happen.</p>