o sufficiently utilize the large dataset $\mathcal {A}$ and $\mathcal {M}$, the model is pre-trained on CTC-based ASR task and MT task in the pre-training stage. 
 Question: What is the attention module pretrained on?
the model is pre-trained on CTC-based ASR task and MT task in the pre-training stage.