The effect of an linear layer?
If you are performing regression, you would usually have a final layer as linear.

Most likely in your case - although you do not say - your target variable has a range outside of (-1.0, +1.0). Many standard activation functions have restricted output values. For example a sigmoid activation can only output values in range (0.0, 1.0) and a ReLU activation can only output positive values. If the target value is outside of their range, they will never be able to match it and loss values will be high.

The lesson to learn here is that it is important which activation function you use in the output layer. The function must be able to output the full range of values of the target variable.