How to verify that a trained neural network is trustworthy?
A definitive and deterministic answer regarding the existence of adversarial samples is only possible by checking the entire sample space, which you said is finite and hence feasible but maybe too computationally expensive.
I would suggest to generate a grid of samples over the entire space to get a somewhat representative subspace and evaluating the model on it. As you suggested, the normal error distribution out of that whould give you a more or less reliable probabilistic statement on future model errors, depending on the size of the subspace I guess.
However, I also am curious if there is a better way to evaluate ones model reliability.