It is important to note that in multi-headed attention, each head acts as an independent entity in the evaluation of relationships in the data. If each attention head can be thought of as an "expert" developing a theory for causal relationships between features and prediction, in this analogy each expert is locked in a separate room independently processing the same data. The final attention-weighted representations are averaged versions of all the experts' conclusions.