Training an RNN to answer simple quesitons
I have thought of various ways of doing this. Such as, as well as having 26 outputs for letters of the alphabet have about 20 more for "repeat the character that is 14 characters to the left" and so on


Creating a system of rules like your example above is the very opposite of training an RNN to perform this task.

If you would like to train an RNN to answer simple questions you would not need to come up with ingenious rules, but with enough training data of the form

question -&gt; answer


Then you could use one of many popular sequence-to-sequence NLP tools to try and learn this behaviour, effectively treating the problem as if it were machine translation between languages.



More broadly, yes, question answering has been done before, it is very popular in fact. It is an active subfield of NLP research and many methods have been developed, some of them involving RNN networks.