For all experiments, the dimensions of word embeddings and recurrent hidden states are both set to 512. 
 Question: Do they use the same architecture as LSTM-s and GRUs with just replacing with the LAU unit?
Yes