We employ a parameterized embedding layer with output dimension 32 and a subsequent dropout on features of rate 0.4; a two-layer LSTM with 256 hidden units each and a dropout of rate 0.6 between recurrent layers; a linear layer with no parameter sharing. Having an additional, penultimate layer with the number of units equalling to the embedding dimension allows parameter sharing, and from our experience, it works better for larger vocabularies. All feed-forward layers are initialized with the Xavier algorithm and recurrent layers with orthogonal initialization. On every iteration, the hidden states are detached from the computational graph and reinitialized to zeros.