Is there an equivalent model to the Hidden Markov Model for continuous hidden variables? I understand that Hidden Markov Models are used to learn about hidden variables $z_i$ with the help of observable variables $\xi_i$. On Wikipedia, I read that while the $\xi_i$'s can be continuous (say Gaussian), the $z_i$'s are discrete. Is this necessary, and why? Are there ways in which I could extend this to continuous domains?
Kalman filter is what you're looking for.
According to Wikipedia:

The Kalman filter may be regarded as analogous to the hidden Markov model, with the key difference that the hidden state variables take values in a continuous space (as opposed to a discrete state space as in the hidden Markov model).