The minibatch size and time span are set to 128 and 64, respectively. The choice of a time span has more effect on final results and we suggest experimenting on values between 64 and 80. The batch sizes between 64 and 128 with the corresponding adjustments on learning rates give almost identical results. The initial and final learning rates are set to 0.001 and 0.0001, respectively, with intermediate updates on every iteration of minibatch SGD (we use the Adam optimizer ) according to the cosine learning scheduler. The gradient clipping radius is set to 10, although with shorter time spans, gradient exploding occurs rarely.