Steps on how to use autoencoders to reduce dimensions
I will repost my answer on Quora here, about the "should I train the autoencoder on the whole dataset" part of the questions.

TL;DR: NO, you always use ONLY the training set for training, even for unsupervised learning with autoencoders!

What is the goal of using a test set ? It is meant to represent data that you model (including both the autoencoder and the random forest) have NEVER seen before.

Autoencoders are unsupervised learning models, but it does not mean that they cannot overfit. They can learn latent codes that do not generalize. Imagine an AE that encodes every data point x1, x2, … as a real number (one-dimensional code):

x1 -> 0.0001, x2 -> 0.0002, etc.

Do you think this autoencoder will generalize well ? No, it just remembered the entire training set and did not learn useful representations. That is why we use regularization in autoencoders, in very different forms : undercompleteness, sparsity, denoising, variational autoencoders (VAE)…

Conclusion, if you include the test set to train your autoencoder, it may in certain cases overfit, your performance estimates will be biased and you lose all the purpose of a test set.

(don’t hesitate to ask if you want me to detail some aspects of my answer or have more questions on this matter)



Concerning the architecture of you AE:


using a sigmoid output activation only makes sense if you data takes values between 0 and 1, as a sigmoid function squashes the output into the [0, 1] range. So either you normalize your data to the [0, 1] range, or you use a linear output activation.
With a sigmoid output activation, you should use a binary cross-entropy loss, which is designed for binary variables following Bernoulli distributions. With a linear output, use the mean squared error loss, which is designed for real-valued output (following Gaussian distributions).
Maybe Adam optimizer will converge faster that Adadelta. Try for yourself.