How to deal with broad and narrow variance within classes in classification tasks
You will want to have many diverse examples of the high variance classes and pick a model that has a high learning capacity. In other words, a large number of samples and a big model to capture the properties of the data.

After training the model, look at the confusion matrix by class for the hold-out dataset to see if the model is learning to generalize for all classes, including the high variance classes.