As our models did not use RL refinement, these results constitute significant gains of 1.34 and 1.12 BLEU score on top of the strong baselines in BIBREF3 . The perplexity scores are also better. On the Google Production dataset, our model achieved 1.01 higher test BLEU score even after training for only one sixth of the time. 
 Question: What improvement does the MOE model make over the SOTA on machine translation?
1.34 and 1.12 BLEU score on top of the strong baselines in BIBREF3 perplexity scores are also better On the Google Production dataset, our model achieved 1.01 higher test BLEU score