The least-mean-magnitude-phase (LMMP) algorithm is useful for complex-valued signal processing applications where control of magnitude and/or phase error information is needed to achieve good overall performance. Due to the highly-nonlinear nature of the update terms in the LMMP and related methods, few convergence and stability results exist to guide step size choices for such algorithms. In this paper, we provide a rigorous stability and convergence analysis of the LMMP algorithm using robustness procedures and give sufficient stability conditions on the magnitude and phase step sizes to guarantee contraction-mapping behavior. We also provide an approximate relation for the steady-state MSE as a function of the step size values. Simulations verify the predictive powers of our analytical results and yield useful insights on step size choices in practice.