Why use a recurrent neural network over a feedforward neural network for sequence prediction?
An RNN or LSTM have the advantage of "remembering" the past inputs, to improve performance over prediction of a time-series data. If you use a neural network over like the past 500 characters, this may work but the network just treat the data as a bunch of data without any specific indication of time. The network can learn the time representation only through gradient descent. RNN or LSTM however have "time" as a mechanism built into the model. The model loops through the model sequentially and have a real "sense of time" even before the model is trained. The model also have "memory" of previous data points to help the prediction. The architecture is based on the progress of time and the gradient are propagated through time as well. This is a much more intuitive way to process time-series data. 

A 1D CNN also will work for the task. An example of CNN in time series data is wavenet, which uses CNN for generating incredibly life like speech using dilated convolution neural network. For whether LSTM or CNN works better, it depends on the data. You should try experimenting with both networks to see which works best. 

Suppose you need to classify a video's genre. It is much simpler to watch it in sequence then seeing frames of it playing randomly in front of your eyes. This is why an RNN or an LSTM works better in time series data.