Notably, disabling the first layer in the RTE task gives a significant boost, resulting in an absolute performance gain of 3.2%. However, effects of this operation vary across tasks, and for QNLI and MNLI, it produces a performance drop of up to -0.2%. 
 Question: How much is performance improved by disabling attention in certain heads?
disabling the first layer in the RTE task gives a significant boost, resulting in an absolute performance gain of 3.2%  this operation vary across tasks