Multidimensional gradient descent in Tensorflow
Tensorflow first reduces your loss to a scalar and then optimizes that.
What does Tensorflow really do when the Gradient descent optimizer is applied to a "loss" placeholder that is not a number (a tensor of size 1) but rather a vector (a 1-dimensional tensor of size 2, 3, 4, or more)?
Is it like doing the descent on the sum of the components?