We have seen some evidence that the linear transformation just discussed is not sufficiently flexible. In three cases we see that training to a single dataset can outperform the multitask model on a given task. This indicates that the multitask model does not have sufficient capacity when training to datasets that yield models that perform worse on that particular task. On the other hand, as one increases the expressiveness of the output trans-formation the amount of information shared between heads will likely decrease, resulting in less information transfer. One must therefore balance this trade off.