Help with understanding why we need an expectation in the loss I am reading the paper SV2P - Stochastic Varaiational Video Prediction (https://arxiv.org/abs/1710.11252). In it the authors use the loss shown in the image:


Why do we need expectation in the reconstruction loss?
I think that the loss is similar to the one used in the VAE paper.
It's because $z$ is a random variable, and the term is a function of $z$.  So, without the expectation, the $\log$ term would be a random variable, and you can't use that as a loss; the loss needs to be a number.  You can use its mean (its expectation).