Is there an alternative to RNNs that doesn't require knowing input history? In an RNN to train it, you need to roll it out, and enter in the history of inputs and the history of expected outcomes. 

This doesn't seem like a realistic picture of the brain since this would require, for example, for the brain to store a perfect history of every sense that comes in to it for many time-steps. 

So is there an alternative to RNNs that doesn't require this history? Perhaps storing differences or something? Or storing some accumulator?

Perhaps there is a way to calculate with RNNs that doesn't require keeping hold of this history?
You don't necessarily need to roll out the inputs to an RNN; doing so makes it easier to optimize computation (if the sequence length is the same each batch), but it's not a necessity. Furthermore, RNNs (and, incidentally, the brain) doesn't necessarily remember the input history as is; rather, the history is encoded via the RNN's cell state (or states, in the case of LSTMs and other RNN cell architectures with multiple states). Neural Turing Machines (NMTs) and Differential neural computers expand on that concept by also using a larger "memory" storage (in the form of a matrix).