Understanding why ensembling only improves marginally
From what you are saying on can make up that you predict scores for each class and that the highest score is your class prediction.

A question you may ask is whether or not the class prediction is affected by the ensembling at all. For the sake of the example consider an example where Model A predicts a class always with a normalised score of 1.0 (i.e. full confidence, recommended reading). Suppose Model A is right 80% of the time. Model B however, is less certain. It places his bets on the top 5 classes that it deems likely with an equal normalised score of 0.2. Averaging the scores results in the same top predicted class as for Model A alone. You could easily check this by calculating the correlations between the predictions of the models with the ensembled predictions.