title: How to use GPU to speed up computation with tf.Keras
body: <p>These days, I failed to use GPU to spped up my new program.
  My GPU is Nvidia 1080Ti, and I have speeded up my program with the GPU before. (As the GPU-util achieve more than 60%, and the time is much shorter than using CPU). So I think there's no problem with my environment set-up. </p>
  
  <p>Compared with my previous programs that working well with GPU, in the new program I defined a complicated loss function with many tensorflow operations. Since the net model is similar, I think the complicated loss leads to time increase. </p>
  
  <p>My code is listed as follows:</p>
  
  <pre><code>import numpy as np
  import os
  from keras.callbacks import ModelCheckpoint, EarlyStopping
  from keras import backend as K
  
  NT, H_num, Nk = 64, 100000, 64
  Input_H = np.load('Input_H10W.npy')
  print(Input_H.shape)
  input_dim = NT * Nk
  Nt = int(NT/2)
  
  model = tf.keras.models.Sequential([
      tf.keras.layers.InputLayer(input_shape=(input_dim,)) ,
      # tf.keras.layers.Dense(4096, activation='relu') ,
      # tf.keras.layers.BatchNormalization(),
      tf.keras.layers.Dense(2048, activation='relu'),
      tf.keras.layers.BatchNormalization(),
      tf.keras.layers.Dense(1024, activation='relu'),
      tf.keras.layers.BatchNormalization(),
      tf.keras.layers.Dense(256, activation='relu'),
      tf.keras.layers.BatchNormalization(),
      tf.keras.layers.Dense(Nt, activation='tanh')
  ])
  
  def channel(y_true, y_pred):
      results, CSI = y_pred, y_true
      a = tf.constant(3.1415926)
      wr = tf.cos(results* a)
      wi = tf.sin(results* a)
      w = tf.complex(wr,wi) #actually W^H 1*16
      mse = 0
  
      for i in range(Nk):
          hr = tf.slice(CSI, [0, NT*i], [-1, Nt])
          hi = tf.slice(CSI, [0, NT*i+Nt], [-1, Nt])
          h = tf.complex(hr,hi)
          equal_H = K.batch_dot(w, tf.transpose(h))
          euqal_H_conj = equal_H - 2j * tf.cast(tf.imag(equal_H), dtype=tf.complex64)
          snr = euqal_H_conj * equal_H
          snr = tf.cast(snr, dtype=tf.float32)
          mse += 1 / (snr/Nt + 1)
      mse = mse/Nk
      return tf.reduce_mean(mse)
  
  
  optimizer = tf.train.AdamOptimizer()
  model.compile(optimizer, loss=channel)
  print(model.summary())
  checkpoint = ModelCheckpoint('./trained.h5', monitor='val_loss', verbose=0, save_best_only=True, mode='min', save_weights_only=True)
  model.fit(Input_H, Input_H, epochs=2000, batch_size=2048, verbose=2, callbacks=[checkpoint], validation_split=0.2)
  model.load_weights('./trained.h5')
  </code></pre>
  
  <p>The result of the program is all right but it runs too slowly. As all thing is normal except the complicated loss function, so I think it is the reason. I monitor the GPU utilization with some software, during the training, it sometimes achieve 30%, but at most time is 0%. And thus my program run very slowly.</p>
  
  <p>So, how can I utilize my GPU to speed my program?
  Thanks in advance and sorry for my English.</p>