Can a character-level Seq2Seq setup learn to perfectly reconstruct structured data like name strings?
If you give it a name, it will probably be almost perfect model, as the number of hidden units are definitely enough to store the raw data of a name. However as a neural network it will still not be perfect. There may also be serious overfitting if you do this. A Seq2Seq model have way more parameters than necessary to just "remembers" all the possible names, as names variants are not a lot.