title: Specify gpu in Tensorflow code: /gpu:0 is always working?
body: <p>I have 3 graphics cards in my workstation, one of them is Quadro K620, and the other two are Titan X. Now I would like to run my tensorflow code in one of the graphics card, so that I can leave the others idle for another task.</p>
  
  <p>However, regardless of setting <code>tf.device('/gpu:0')</code> or <code>tf.device('/gpu:1')</code>, I found the 1st Titan X graphics card is always working, I don't know why.</p>
  
  <pre><code>import argparse
  import os
  import time
  import tensorflow as tf
  import numpy as np
  import cv2
  
  from Dataset import Dataset
  from Net import Net
  
  FLAGS = None
  
  if __name__ == "__main__":
      parser = argparse.ArgumentParser()
      parser.add_argument('--foldername', type=str, default='./data-large/')
      parser.add_argument('--batch_size', type=int, default=100)
      parser.add_argument('--num_epoches', type=int, default=100)
      parser.add_argument('--learning_rate', type=float, default=0.5)
  
      FLAGS = parser.parse_args()
      net = Net(FLAGS.batch_size, FLAGS.learning_rate)
  
      with tf.Graph().as_default():
          # Dataset is a class for encapsulate the input pipeline
          dataset = Dataset(foldername=FLAGS.foldername,
                                batch_size=FLAGS.batch_size,
                                num_epoches=FLAGS.num_epoches)
  
          images, labels = dataset.samples_train
  
          ## The following code defines the network and train
          with tf.device('/gpu:0'): # &lt;==== THIS LINE
              logits = net.inference(images)
              loss = net.loss(logits, labels)
              train_op = net.training(loss)
  
              init_op = tf.group(tf.initialize_all_variables(), tf.initialize_local_variables())
              sess = tf.Session()
              sess.run(init_op)
              coord = tf.train.Coordinator()
              threads = tf.train.start_queue_runners(sess=sess, coord=coord)
              start_time = time.time()
              try:
                  step = 0
                  while not coord.should_stop():
                      _, loss_value = sess.run([train_op, loss])
                      step = step + 1
                      if step % 100 == 0:
                          format_str = ('step %d, loss = %.2f, time: %.2f seconds')
                          print(format_str % (step, loss_value, (time.time() - start_time)))
                          start_time = time.time()
              except tf.errors.OutOfRangeError:
                  print('done')
              finally:
                  coord.request_stop()
  
              coord.join(threads)
              sess.close()
  </code></pre>
  
  <p>Regarding to the line "<code>&lt;=== THIS LINE</code>:"</p>
  
  <p>If I set <code>tf.device('/gpu:0')</code>, the monitor says:</p>
  
  <pre><code>|-------------------------------+----------------------+----------------------+
  | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
  | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
  |===============================+======================+======================|
  |   0  Quadro K620         Off  | 0000:03:00.0      On |                  N/A |
  | 34%   45C    P0     2W /  30W |    404MiB /  1993MiB |      5%      Default |
  +-------------------------------+----------------------+----------------------+
  |   1  GeForce GTX TIT...  Off  | 0000:04:00.0     Off |                  N/A |
  | 22%   39C    P2   100W / 250W |  11691MiB / 12206MiB |      8%      Default |
  +-------------------------------+----------------------+----------------------+
  |   2  GeForce GTX TIT...  Off  | 0000:81:00.0     Off |                  N/A |
  | 22%   43C    P2    71W / 250W |    111MiB / 12206MiB |      0%      Default |
  +-------------------------------+----------------------+----------------------+
  </code></pre>
  
  <p>showing the 1st Titan X card is working.</p>
  
  <p>If I set <code>tf.device('/gpu:1')</code>, the monitor says:</p>
  
  <pre><code>|-------------------------------+----------------------+----------------------+
  | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
  | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
  |===============================+======================+======================|
  |   0  Quadro K620         Off  | 0000:03:00.0      On |                  N/A |
  | 34%   45C    P0     2W /  30W |    411MiB /  1993MiB |      3%      Default |
  +-------------------------------+----------------------+----------------------+
  |   1  GeForce GTX TIT...  Off  | 0000:04:00.0     Off |                  N/A |
  | 22%   52C    P2    73W / 250W |  11628MiB / 12206MiB |     12%      Default |
  +-------------------------------+----------------------+----------------------+
  |   2  GeForce GTX TIT...  Off  | 0000:81:00.0     Off |                  N/A |
  | 22%   42C    P2    71W / 250W |  11628MiB / 12206MiB |      0%      Default |
  +-------------------------------+----------------------+----------------------+
  </code></pre>
  
  <p>showing that the two Titan X cards are working, not the 2nd Titan X alone.</p>
  
  <p>So any reason behind this and how to specify the gpu I want my program to run in?</p>