We tuned the following hyper-parameters on the validation set: initial leaning rate (5 • 10 -3 , 1 • 10 -3 and 5 • 10 -2 ) and number of layers . The rest of the hyper-parameters were set as in . We trained our models for around 750 epochs until the log-log plot of the loss was converged taking about 10 days to train in a single GPU.