What is rotor damping and how does it affect helicopter stability?
I will try to answer these questions in terms of structural dynamics.
Please correct me if I say dumb things on other parts of my answer.
You ask about damping. Damping must be considered when dealing with structural dynamics, or vibratory phenomena. In a helicopter rotor, multiple phenomena can lead to vibrations. For instance :

Differences in blade properties (mass mostly, but also stiffness)
Rotor imbalance due to machining uncertainties
Bearings
Engine, which may produce torque oscillations of some sort
Gearbox, which may also produce oscillations (look for gear mesh vibrations)
Helicopter operations (if you have quick flight trajectory modifications, this may lead to increased or reduced lift needs, which induce torque oscillations in the drive line)

So long story short, multiple phenomena can excite the rotor.
As any structure, a helicopter has natural vibration patterns called natural modeshapes, associated with natural frequencies.
However, if you excite a component enough on its natural frequency, this may result in a fatigue failure, hence the utmost importance of damping.
As far as I know, the most visible technology used to increase damping on a helicopter rotor are the blade root dampers. Multiple technologies exist, however they seem to damp the same phenomenon : blade edgewise motions or so-called lead-lag motions or hunting motions.
If you want to have a deeper understanding of these issues, have a look at this article for instance. This page may be helpful as well.
The first paper addresses semi-active dampers to control the blade edgewise bending motions in order to reduce the constraints on the hub. Basically, if the blade was mechanically fixed to the hub, large constraints could occur due to vibration loads, leading to for instance fatigue failure. In the configuration presented in the paper and reproduced here,

the liaison between the hub and the blade is a cylindrical liaison, allowing in-plane rotational motion. The (stiffness-)damper system is then used to absorb the kinetic energy of the blade in order to limit the loads on the hub, which allows to reduce its overall weight.
The second paper provides some more general concepts on the technologies involved in helicopter rotors. In particular this here

is a good schematic of a full articulated rotor.
Other rotor technologies exist : semi-rigid or rigid rotors but they use a different strategy for vibration reduction and blade-hub liaison.