Tensorflow no_grad concept
If you don't want to train certain Variables in TensorFlow you can achieve this behaviour by adding trainable=False to Variables.
I know with pytorch you can turn off training by calling eval() on your model. 
Also you can set requires_grad=False. 
How can you ensure that a TensorFlow element is not modified during training?