title: how can I change tensorflow optimizer during training
body: <p>I read before that sgd generalize better but adam converge fast in early stage of training.
  So, I want to change my optimizer after some steps. But code like below raised preconditionederror. If anyone knows how can I change optimizer during training?</p>
  
  <pre><code>for i in range(config.train_steps):
      _, l, s = sess.run([train_op, loss, summary])
  
      if i % 100 == 0:
          saver.save(sess, os.path.join(args.model_dir,
                                            'model.ckpt'))
          writer.add_summary(s, i)
          print('loss: ', l)
      if i == 1000:
          train_op = tf.train.GradientDescentOptimizer(learning_rate)
      coord.request_stop()
      coord.join(threads)
  </code></pre>