Adaptive fiiler (LMS) step size bounds I am reading a book on adaptive filters.
For the optimal filter (Wiener), you can analytically obtain the step size for the gradient descent ($0&lt;\mu&lt;\frac{2}{\lambda_{max}}$).
This is not scary, because for the algorithm itself, the correlation matrix is ​​also needed, at the same time it is possible to find the eigenvalues ​​(although in practice it will probably take a long time).
When switching to adaptive filtering (the first considered LMS algorithm), averaging is abandoned (you may not know the statistical characteristics of the process, great!). Now the algorithm uses just instantaneous signal values. But at the same time, the requirements for stability in the book indicate the same. But where can we get them if we have abandoned the knowledge of the correlation matrix, how to correctly implement the algorithm?
You can analyse the input online, and get some rough estimates on its power, energy, pdf, autocorrelation matrix, etc. and also its eigenvalues of (time-varying) autocorrelation matrix. Then you may estimate the associated upper bound for the step-size of LMS.
Nevertheless, finding the optimal step-size in applications within unknown environments is not easy. So many of the LMS variations are based on different approaches for efficiently, and reliably finding the best step-sizes.