Arden's rule expressed as matrix algebra
The notation is probably defined in the paper or in some standard reference, but one can guess that
$$
A^* = (I - A)^{-1}.
$$
This definition satisfies the identity $A^* = I+AA^*$, since
$$
I+AA^* = I-((I-A)-I)A^* = I-(I-A)A^*+A^* = I-I+A^* = A^*.
$$
Presumably a quasiregular matrix $A$ is one for which $I-A$ is invertible, perhaps with some additional assumptions.