how to design sigma(std) vector to have values above > 0 in VAE autoencoder I am trying to understand how to design mu and sigma vector in VAE autoencoder, as we know std value can not be larger than 0. If I use linear function, the value can be lower than zero, so I think Relu is the only activation function to apply in order to get sigma (Std) vector larger than zero. Would this be correct understanding?
Yeah, almost, except that Relu might cause problems ones it goes to zero (having zero variance is a little odd and the gradient goes to zero as well). Usually you'd go with

$$ \sigma = \exp(\rho) $$

where $\rho$ is coming from the encoder.