Imposing physical constraints (previous knowledge) in a neural network for regression I'm trying to train a neural network to do a multiple non-linear regression $y=f(x_i), i=1,2…N$. So far it works good (low MSE), but some predictions $y$ are “non-physical”, for instance for our application it is known  from first principles that when $x_2$ increases, then $y$ also has to increase ($dy/dx_2&gt;0$), but in some instances the neural network’s output doesn’t comply with this constraint. Another example is that $y + x_5 + x_7$ should be less than a constant $K$

I thought about adding a penalty term to the loss function to enforce these constraints, but I am wondering if there is a "harder" way to impose such a constraint (that is, to ensure that these constraints will always hold, no only that non-physical predictions will be penalized)
Some ideas out the top of my head:


In the case of $dy/dx_2&gt;0$ you could compute the gradient using the chain rule and limit the weights so that the constrain holds
In the case of $y + x_5 + x_7 &lt; K$ you could use a clipping function on the output layer?