Are on-line backpropagation iterations perpendicular to the constraint?
Answer by Theo Bandit at maths stackexchange


  If you choose two points $(w_1, w_2), (v_1, v_2)$ along this line, then
  $$(x_1, x_2) \cdot ((w_1, w_2) - (v_1, v_2)) = x_1 w_1 + x_2 w_2 - (x_1 v_1 + x_2 v_2) = y - y = 0.$$
  That is, the direction $(x_1, x_2)$ is perpendicular to any vector lying along the line, i.e. $(x_1, x_2)$ is normal to the line.