Classification when some classes are dependent
I'm afraid there is no exact answer to your question.
In the scenario you described above male and female mammals are very similar, similar enough that the average person wouldn't notice the difference.
In such a case, I would suggest going with the first option of classifying into 10 classes of mammals and then building 10 classifiers of male-female.

This of course isn't an answer that applies to all cases, it is strongly dependent on a few factors, mainly on the classification scenario at hand.
The size of the dataset is also a significant parameter.
Unfortunately I don't think there is a methodological way to address this issue, only experience can point you towards the best solution.