Title: HyperENTM: Evolving Scalable Neural Turing Machines through HyperNEAT,
Abstract: Recent developments within memory-augmented neural networks have solved
sequential problems requiring long-term memory, which are intractable for
traditional neural networks. However, current approaches still struggle to
scale to large memory sizes and sequence lengths. In this paper we show how
access to memory can be encoded geometrically through a HyperNEAT-based Neural
Turing Machine (HyperENTM). We demonstrate that using the indirect HyperNEAT
encoding allows for training on small memory vectors in a bit-vector copy task
and then applying the knowledge gained from such training to speed up training
on larger size memory vectors. Additionally, we demonstrate that in some
instances, networks trained to copy bit-vectors of size 9 can be scaled to
sizes of 1,000 without further training. While the task in this paper is
simple, these results could open up the problems amendable to networks with
external memories to problems with larger memory vectors and theoretically
unbounded memory sizes.