Nonlinear adaptive filtering techniques for system identification (based on the Volterra model) are widely used for the identification of nonlinearities in many applications. In this correspondence, the improved tracking capability of a numeric variable forgetting factor recursive least squares (NVFF-RLS) algorithm is presented for first-order and second-order time-varying Volterra systems under a nonstationary environment. The nonlinear system tracking problem is converted into a state estimation problem of the time-variant system. The time-varying Volterra kernels are governed by the first-order Gauss-Markov stochastic difference equation, upon which the state-space representation of this system is built. In comparison to the conventional fixed forgetting factor recursive least squares algorithm, the NVFF-RLS algorithm provides better channel estimation as well as channel tracking performance in terms of the minimum mean square error (MMSE) for first-order and second-order Volterra systems. The NVFF-RLS algorithm is adapted to the time-varying signals by using the updating prediction error criterion, which accounts for the nonstationarity of the signal. The demonstrated simulation results manifest that the proposed method has good adaptability in the time-varying environment, and it also reduces the computational complexity.