What is the mean in the variational auto-encoder? Here's a diagram of a variational auto-encoder.



There are 2 nodes before the sample (encoding vector). One is the mean, one is the standard deviation. The mean one is confusing. 

Is it the mean of values or is it the mean deviation?

$$\text{mean} = \dfrac{X_i+..+X_n}{N}$$

$$\text{mean deviation} = \dfrac{[X_i|+..+|X_n|}{N}$$
Kind of neither, although leaning towards the first definition of the Mean as a simple average of values.

It's a distribution parameter of the Gaussian, so it's the expected average of samples as the number of samples approaches infinity.

The distrinction is that you could draw three samples, -2, 0 and -1, from a Standard Normal - the mean of samples would be -1, but the distribution mean is still 0.

From a network architecture PoV, it's a learnable transformation applied on top of samples from a Standard Normal, so you only need to learn the transformation and get sampling for 'free'.