What is an "innovation filter"? I'm a math postgrad student working through a paper on eigenvalue decompositions of matrices of FIR filters (used for stuff like total decorrelation, convolutive mixing, MIMO). Towards the beginning, when describing their signal model, they use the phrase:


  If the PSD of the $l^{th}$ source is generated by a stable and causal innovation filter $\text{F}_{l}(z)$, ...


What does this mean? I understand a stable filter is one whose coefficients $a[n]$ approach 0 as $n \rightarrow \infty$, and a causal filter is one which does not use values from the future (i.e. no negative time lags). I have read that the innovation in terms of stochastic processes is defined as the difference between the actual next value of a time series, and the value given by an optimal prediction based on the information already available.

I cannot, however, find any description of an innovation filter. I have a mathematics background rather than a DSP/engineering background, and so sometimes some of the terminology trips me up, which is why I was hoping DSP stack exhange could kindly help!
An innovations filter of a WSS process $X(t)$ is a causal and stable minimum phase filter that can be used to generate $X(t)$ from a white noise input $N(t)$:

$$X(t)=\int_0^{\infty}h(\tau)N(t-\tau)d\tau\tag{1}$$

where $h(t)$ is the impulse response of the innovations filter.

The (causal and stable) inverse filter of $h(t)$ is called the whitening filter of $X(t)$. Its response to the input $X(t)$ is white noise $N(t)$.

A process that can be represented by $(1)$ is called a regular process. Note that its power spectrum is given by

$$S(\omega)=|H(\omega)|^2\tag{2}$$

where $H(\omega)$ is the Fourier transform of the impulse response $h(t)$ of the innovations filter (assuming that $E\{N(t)N(t+\tau\}=c\delta(\tau)$ with $c=1$). The requirement of causality of the innovations filter implies that the power spectrum $S(\omega)$ of $X(t)$ satisfies the Paley-Wiener condition

$$\int_{-\infty}^{\infty}\frac{|\log S(\omega)|}{1+\omega^2}d\omega\lt\infty\tag{3}$$

This implies that $S(\omega)$ cannot contain spectral lines, and it cannot be band-limited. Unlike a singular process (consisting of spectral lines), a regular process cannot be parameterized by a finite set of random variables and it is not predictable, i.e., it is not completely determined in terms of its past.

Reference:
Probability, Random Variables, and Stochastic Processes, A.Papoulis, Athanasios 1965. McGraw-Hill.