Metric to punish positive errors I'm involved in a ship transit time prediction project. Is about prediction the time that a ship's cargo takes to go from port A to B in order to contract the fastest carrier company and tell the client how long it going to take, because a several times a the carrier company takes more or less (most of the times more).
We have historical database with predictive variables and stuff, the point is if the real transit time of cargo is 10 days is worse to the model estimate 9 days than eleven days, even that the absolute error is 1.
So a need a custom metric to punish positive errors more than negative ones and punish absolute large errors as well, calling error = actual - predicted.
I thought in using a weighted mean squared error, but I don't know how to balance the weights. Do you guys may help me? Do you know some other metrics that meet that requirements?
Thanks very much!
How about using the difference of an exponential and a linear or a polynomial functions?
For example, if $x = actual - predicted$, then
$$L = w_1 (e^x-1) - w_2 x$$
The cost of error increases faster for positive values than for negative ones, and is a differentiable function of $x$. You can tweak $w_1$ and $w_2$ depending on your typical errors and your learning algorithm. This is an example, for $w_1 = w_2 = 20$: