CNN for subsets of a dataset - how to tune hyperparameters
It depends on how you choose the subset of the dataset. One required assumption about data for any Machine Learning model to generalize well on unseen data is that the data must come from the same statistical distribution.

So, if you select the subset at (uniformly) random, it's quite sure that your model will continue to perform well. Otherwise, I think it's better to fine-tune the big model on the subset for better performance.