How can I prove bottleneck layer of my CNN auto encoder contain useful information? I am using CNN autoencoder to create a state representation layer which I will later be feed into my Reinforcement Agent. So I trained my CNN autoencoder and it is giving nice state representations. But I have following questions,


Can my autoencoder layer be overfitted
If there's a overfit will it cause rubish information in my bottleneck layer?
Yes to both of your questions. Your autoencoder can overfit and this will cause your bottleneck to store useless information (besides any useful information it already stores).

Some ways to prevent this is:


Find a larger dataset, or augment the current.
Add noise to the input (see de-noising autoencoders).
Regularization (e.g. early stopping, sparsity constraints)