Is there a way to make my neural network discard inputs with bad results from learning? What I want to achieve is this: If my desired outputs are [1, 2, 3, 4] I would rather have my network produce this output:
[0.99, 2.01, 999, 4.01]
than say this:
[0.94, 1.88, 3.12, 4.1]
So I'd rather have a few very accurate outputs and the rest completely off, than have them all be decent but no more than that. My question is, is there a known way to do this? If not, would it make sense to remove the inputs that produce poor outputs, and redo the learning phase?
I assume [1, 2, 3, 4] are the desired outputs for different examples for a regression task. Sound like you need a different loss function. From your description it seems you don't care how big the error is if it's bigger than some value. Try the Huber loss(in Pytorch and TensorFlow). Examples that are far from the expected value won't produce big gradients (: