Why does my "entropy generation" RNN do so badly?
I think there are two problems with your network.
The first one, always having very similar outputs, is the rather simple one. As it seems, your network suffers from the so-called, very common Mode Collapse problem. The attached link provides both an explanation and some potential remedy to that problem.
The second problem is more fundamental. You say that you want your network to produce random numbers. Or numbers that at least appear as such. However, once training is finished, your model is going to be a static function which will not change any further. Given the same input x, it will always produce the same output y. Consequently, unless the inputs to your network contain some randomness already or are, at least, always slightly dissimilar, you will not end up having a random generator. So, whether that is going to be useful will depend on your usecase. But if you make sure that the true random variable (like date&amp;time) serves as input to the RNN and the RNN just translates this into some different format, that might work again. Just keep in mind that randomness can never arise out of a trained model.