Why might a neural network consistently underestimate its target? I have a neural network (MLP) that is consistently underestimating the target variable on the validation set, test set, and on the training set (by about the same amount as on the validation set and test set). In other words, the sum of the entries in the neural network regression output column (Y_hat) is something like 10% less than the sum of the entries in the target variable column (Y). The entries in the target column are greater than or equal to zero, with a decent number of zero entries. The right-side tail of the distribution of the target variable is long.

Fitting the neural network using many random seeds consistently leads to similar results (the neural networks are all biased in the same direction and by a similar amount).

The problem seems to be fairly robust to changes in important hyperparameters, including:
- early stopping
- learning rate schedule
- model complexity
- regularization (dropout, batch norm)

The problem is less severe for low-capacity neural networks.

Does anyone have any ideas as to why this persistent underestimation might be happening?
Consistently underestimating target could be due to the distribution of the target variable. If the target distribution has a negative-skew (i.e., a long tail towards lower values), then the neural network is just pattern matching to minimize those errors. For example, if the network has a squared loss function then large estimation errors in the tail are weighted more heavily than small estimation errors in the head, even though there are fewer data points in the tail. The model is trying to minimize the overall loss function and is "cheating to win" by consistently underestimating.