Restrictions on my skewed validation data
I suggest using your whole validation set but providing class-specific metrics. E.g., AUC only considering validation rows with targets in a single class. That will help you see if you're underperforming for a specific class. You can then say "This model earned 0.xxx AUC on the whole validation set and no worse than 0.yyy for any single class."