Fixed effects or random effects in RNN
This is an interesting question. Fixed effects have been developed in a linear model world (OLS-like). So maybe the twist in a highly non-linear world (like NN) is a little different. As far as I know, NN has not yet had too much impact in econometrics. So it seems that there still is need to bridge some empirical concepts. 

Let's think about fixed effects (FE). FE uses the "within variation" of each subject to which a FE is applied to estimate something. So what is the problem with deriving this within variation (the change from time t=1 to 2, e.g. x2i - x1i) and plug it into NN manually. Remember that an NN without any hidden layer is just a linear like model. If you do something like this you may probably not need a LSTM layer since all that is relevant in this setting is the change from t1 to t2. Normal dense layers will do, I guess.

Another alternative would be to use "dummy FE", so just plug in one dummy for each subject to be controlled by fixed effects. This data representation could probably work well in an LSTM setting. However, I think we cannot strictly speak of a fixed effect model in this case, since - by the nature of NN - there is no "separate constant term" (like in OLS) for each subject. But is this really a problem? I don't think so, because what you still do in the end is, to control for each subject separately. 

A more fundamental question is what you gonna do with the results. FE in economics are primarily used in causal modelling (FE should make sure that certain model assumtions are met). In NN - generally a predictive modelling setup - there are way too many "coefficients" (weights) to make a meaningful causal statement. So I guess that its not so much about the strict econometric meaning of FE, but more about adequate control for FE in NN. In this case, adding dummies for each subject should be a working solution.