How would AI be able to self-examine?
Several AI systems will come up with a level of confidence to the solution found. For example, neural networks can indicate how relatable is the input problem to the ones it was trained with. Similarly, genetic algorithms work with evaluation functions, that are used to select best results, but depending on how they're built (the functions), they can indicate how close the algorithms are to an optimal solution.

In this case, the limit to when this is acceptable or not will depend on a threshold set beforehand. Is 50% confidence good enough? Maybe it's ok for OCR apps (spoiler: it's not), but is it for medical applications?

So yes, AI systems do currently have the capacity of determining if they're performing well or not, but how acceptable that is is currently based on the domain of the problem, which currently stands outside of what is built into an AI.