Information Visualization: which parameters should a data scientist plot to best explain the work behind a Neural Network?
There are different ways to explain your findings but associating the target with the inputs provided is tough and only a few architectures like Decision Trees do it well.

If you are trying to answer the question &quot;why is my neural network missing out on a few samples from my validation set?&quot; then its going to be a tough but possible road ahead. There are many good resources to interpretable machine learning

If you are trying to compare your model's (CNN) performance to other
architectures on the same data set, then looking at Receiver Operator Characteristics is a good point to start.

If you're having an unbalanced data set, then the F1 Score is a good thing to have look at. Being a classification problem (which is my assumption from your question), you can indicate the performance by a variety of derived quantities.