How to tell my neural network that I care much more about precision than recall? I am training a neural network for a multilabel classification problem, so my last layer consist of n_classes sigmoid neurons.

Now, I know that it is impossible to nail the learning task both because of noise and lack of predictable power in my features. However, I am assuming that the predictable power should be enough to make good predictions at least for some data points. Even if these cases are just a few, I would be happy if the neural network gets at least those with certainty (high precision) and misses the grand majority because of their difficulty (low recall).

Of course, I could train the model normally and then set that threshold that matches my goals, but this doesn't convince me. I would like to introduce this goal in the training process. If some points are hard to classify I want my neural network to discover that during training, forget about them and focus on the other more doable cases.

So far my only idea is to make training robust towards outliers via label smoothing.

Any other ideas to influence the model during training like this?
I think you have two options that don't exclude each other:


Check precision/recall metrics constantly during training, on Validation data. In this way, during experimentation and hyperparameter tuning, you will choose models that are more favourable for precision rather than recall.
Customize your loss function to correct the model in the right way. The problem is that precision, recall, F1 score, they are all non differentiable functions, i.e. they cannot enter a Loss function as they are. For that reason, some tried to approximate these metrics in a differentiable way. One example is the soft-F1 score, as explained in this article. If you read the function code, you will find differentiable approximations of TP, TN, FP, FN's. You can copy those lines to customize your personalized pseudo-precision Loss function.