In these plots we find that the models will distribute the correlation across all of the first 5 PC's. Generally the transformer model is observed to make use of the first PC more often and this is expected to be because it does not feature a length based correlation in the first PC.