What Non-linearities are best in Denoising RNN Autoencoders and where should the go?
I have some familiarity with Recurrent encoder-decoder archtectures, especially with seq2seq models. This is a standard implementation:

from tensorflow.keras import Sequential

from tensorflow.keras.layers import Bidirectional, LSTM
from tensorflow.keras.layers import RepeatVector, TimeDistributed
from tensorflow.keras.layers import Dense

from tensorflow.keras.activations import elu, relu


seq2seq = Sequential([

    Bidirectional(LSTM(len_input), input_shape = (len_input, no_variables)),

    RepeatVector(len_input),

    Bidirectional(LSTM(len_input, return_sequences = True)),

    TimeDistributed(Dense(100, activation = elu)),

    TimeDistributed(Dense(1, activation = relu))

])


where len_input is the length of the input trends, and np_vars the number of variables that compose each observations (if it's a univariate series, set it to 1). The choice of Bidirectional() wrappers is optional.

The model works like this: The input LSTM layer encodes the sequential information to a vector containing a latent representation. This vector is replicated by the RepeatVector() layer, its signal is then calculated using Dense() layers, and distributed for each timestep prediction sequentially (by the TimeDistributed() wrapper). You don't worry about non-linearities at this point, since this is learned by internal LSTM states and by Dense() vectors. Fancier implementation would require stacked Recurrent layers, or deeper feed-forward blocks.

If your goal is to build a Denoising Autoencoder you should pair each input series to its "clean" counterpart, so that the Network learns how to reduce noise.