Autoencoders and information compression
In the other situation when in your notation $k&gt;&gt;d$, in one of our papers we did analyze one such autoencoder in details to be able to say something about what you are asking. Does our Theorem 3.2 here, https://arxiv.org/pdf/1708.03735.pdf help? It took us about 15 pages of laborious calculation to get this - hopefully its helpful! :D  

The "??" in the statement is a LaTeX error. It was meant to refer to the previous Theorem 3.1.