Multi-task learning, finding a loss function that "ignores" certain samples
I seem to have a workable solution using this minor tweak to square loss:

$L=(y-\hat{y})^2 \cdot \left(\frac{y}{y + \epsilon}\right)$

Where $y$ is the label and $\hat{y}$ is the prediction, and $\epsilon=1e-6$ is to avoid a divide by zero error (it's ~ the smallest number within 32 bit precision)

In tensorflow:

epsilon = 1e-6
loss = tf.reduce_sum(
          tf.mul(tf.square(prediction - label),
                 tf.div(label, label + epsilon))
)


The second term is 0 when the label is 0 (e.g. loss is 0 and no gradient is added for that sample), and it'll round to 1, or near enough, when the label is non zero. As long as the range of labels does not cross 0 this should work, and that's a simple matter of scaling the labels if it's not the case by default.