KL divergence in VAE
You are right that the output of your encoder neural network is not a random variable, this is the mean $\mu$ and the standard deviation $\sigma$ of your latent random variable. For example if the output of your encoder is $\sigma = 1$ and $\mu = 0.5$, your latent random variable will be normal with mean 0.5 and standard deviation 1. Theb, you can calculate the KL divergence between your random variable and the prior for the latent random variable which is $\mathcal{N}(0, 1)$ in many tutorials.