Why do recurrent layers work better than simple feed-forward networks?
The first reason is the number of parameters. The former case that you've mentioned, for each neuron there should be corresponding entries that would increase the number of training parameters. The other reason is that by employing simple feed-forward neurons you are somehow discarding the temporal information of your data which means you are discarding the sequence information in your data. This is somehow like the spatial data which is obtained by convolutional layers in CNNs.