MSE relevance as a metric when errors < 1 I'm trying to build my first models for regression after taking MOOCs on deep learning. I'm currently working on a dataset whose labels are between 0 and 2. Again, this is a regression task, not classification.
The low y values imply that the loss for each sample is quite low, always &lt; 1. My question is then about the relevance of mse as a metric in such a case : since the loss is &lt; 1, squaring it will result in an even smaller value, making the metric value drop very rapidly. In this case, would it be more relevant to use mae ? Or should I multiply the y values so that the order of magnitude of a sample loss would be &gt; 1.
I found this nice article about regression metrics, but didn't find the answer in it. Thanks for your help.
I'd use relative RMSE
$\sqrt{\frac{1}{n} \sum \frac{(Preicted - True)^2}{True^2}}$.
In this case, close to 0 implies a good model, regardless of the scale of the true values.
Similarly, you can try relative MAE.