why do transformers mask at every layer instead of just at the input layer? working thru the annotated transformer, I see that every layer in both the encoder (mask paddings) and decoder (mask padding + future positions) get masked. Why couldn't it be simplified to just one mask at the first layers of encoder and decoder?
If the masking were only applied in the first layer, the self-attention in the subsequent layers would bring to each position information from future tokens.
Let's break it down with numbers:

At layer $i$, if causal masking is applied, the output at position $t$ contains information about layer $i-1$ at positions $1..t-1$, that is, $L_{i,t} = f_i(L_{i-1,1},...,L_{i-1,t-1})$.

If no causal masking is applied, then the output at position $t$ contains information about layer $i-1$ at all positions in the sequence of length $T$, that is, positions $1..T$ $L_{i,t} = f_i(L_{i-1,1},...,L_{i-1,T})$

If causal masking is applied at layer 1 (the first layer) but not at layer 2 or 3, we obtain that for position t at layer 3 we would have: $L_{3,t} = f_3(L_{2,1},...,L_{2,T}) = f_3(L_{2,1},...,f_1(L_{1,1},...,L_{1,T}))$, which means that position $t$ contains information from future tokens, as $T &gt; t$.


Note: The original answer was wrong and was completely edited. To check the original answer, refer to the post timeline.