How to identify possible false negatives in labels? I am working on a drilling difficulty prediction project using machine learning method. The dataset has lots of geological data as the main features. It also has total drilled depth of each hole.
Not surprisingly, the holes that are seeing drilling difficulty had significantly longer drilled depth than the ones that did not have difficulty. However, the geological difficulty should not be dependent on the drilled length:  Based on particular type of rock, difficulty is always there no matter if it is drilled or not. If we drilled the holes that didn't have difficulty longer, I am sure lots of them will see difficulty eventually.
Having said that, the labels probably have a lot of false negatives (because the difficulty hasn't been encountered yet). How do I identify these false negatives in the labels?
I think your problem is not with identifying the false negatives, it's with defining the classification problem properly.
If I try to phrase what you're trying to predict, it would be something like this: &quot;Given the data about a drilling point, what's the probability of encountering a difficulty up to depth $x$&quot;.
So, if you have a hole which didn't reach that depth $x$, it's not a false negative - it's just a missing label (no test... yet).
If you want to make your prediction based on the information known before drilling (or, at least, before drilling up to a given depth $z$) - limit your model to that information.