Performance of CNN based deep models with number of classes
You might want to have a look at the hierarchical classification idea I described in my masters thesis.

In short:


If I had a problem where I needed to distinguish 42 breeds of dogs and 30 breeds of cats, I would very likely just create one classifier for the $42+30=72$ classes. Making a two-step approach (first cat vs dog, then cat-breed / dog-breed) seems not to give better results and for sure is more complicated
Analysis of errors becomes simpler the less classes you have
A single accuracy score for more than a handfull of classes / skewed data is not meaningful. For example, if you have two models with both 90% accuracy, one might be way better than the other one.