How can I provide an answer to Neural Network skeptics?
Neural networks are essentially a black box, especially big ones. You could know even how it is designed and how it is training, but you really do not know how it is working in the end. In my work lots of people want to understand the model instead of using "black box" models. This is the reason why companies choose to use linear regressions and polynomial models instead of using stronger machine learning algorithms, like LightGBM and Neural Networks.

I never found a true answer to this question. Some engineers are taught that you cannot use models that you cannot understand. Therefore every model that is a "black box" is not usable for them. This means that most of machine learning models are magic and heresy for them. Though take this with a grain of salt, this is my subjective experience. Sometimes as the time passes these people are more willing to use data science methods because it becomes mainstream. They start to trust the methods, because others use them.

The situation is different on the higher level. For high-tier managers it matters less how to interpret the model, but more what results it could give you. They are more willing to try, especially if there is a hype of something, like "artificial intelligence", "data science".

As a result, I could only give you an advice to find some good support higher in the hierarchy of the company. Someone who believes in data science more and who has more power in the company.

In data science community the performance of the model on the test dataset is one of the most important things people look at. Just look at the competitions on kaggle.com. They are extremely focused on test dataset and the performance of these models is really good.

The only problem with performance on the test dataset is that it depends on the data in the test dataset. If in real life you will have completely different data that will be outside of the bounds of the test dataset, then the test dataset will not be able to give a good approximation of the performance of the model in real life.