How to implement coordinate descent using tensorflow?
Not really possible. TF's backprop calculate gradients across all variables based on the values of the other variables at the time of forward prop. If you want to alternate between training w and b, you would unfreeze w and freeze b (set it to trainable=False), forwardprop and backprop, then freeze w and unfreeze b, and forward prop and back prop. I don't think that'd run very fast since TF isn't really design to switch the trainable flag on every mini batch.
For example, for a simple linear model y=wx+b where x and y are input and output respectively, w and b are training parameters, I am wondering, in every epoch, how can I update b first and then update w?