Loss does not decrease after modifying prediction with torch.floor()
For the following code, the loss decreases.
loss_function=nn.MSELoss()
loss=loss_function(pred,label)

But, the loss remains completely unchanged if I change the pred by floor function. I checked the parameters after opt.step(), they are not changing.
loss_function=nn.MSELoss()
loss=loss_function(torch.floor(pred),label)

Why this might happen?
My guess: This torch.floor(pred) operation breaks the computation graph. Other &quot;real&quot; mathematical operation like pred*3 (for example) does not break the computation graph.

It's not breaking the computation graph, the gradients are zero so your steps have no effect.
Consider the plot of floor(x) shown below. Notice that the function is discontinuous so it's technically not differentiable at whole numbers. Also, for every point where it is differentiable it's a flat function. In other words the derivative is zero almost everywhere. PyTorch simply assigns the gradient of floor to be zero everywhere, since there's really no alternative other than to raise an exception. This implies that, regardless of the value of the loss, the gradients of your loss function w.r.t. your parameters will be zero (following the chain rule/backprop). Therefore, any gradient descent based optimizations will have no affect on the model parameters.

(Image source: Wikipedia)
