Why are low-dropout (LDO) voltage regulators unstable?
A LDO is a control loop. And like all control loops, there is always room for instability. 

So how do you make a control loop stable ? 


You provide sufficient phase margin (difference in phase from when the gain crosses the 0 dB axis and 180.
The slope of open loop plot should be -20db/dec when crossing the 0dB axis
Provide sufficient gain margin


If you we look at a typical open loop response of an LDO, it may look like this



There are a number of poles.


Error amp pole - a pole due to the amplifier
Load pole - pole due to output capacitor and load
Parasitic pole - usually within the pass element (not shown in this image).


There is also one zero in this image. 


ESR Zero - a zero due to the output capacitor


If you look at point 2 of a stable loop, it says that the slope should be -20db/dec.

Well, what if...the zero was never there. That means that the slope when it hits 0db, is -40db (due to the two previous poles). Instability. 

Adding a zero before the 0db axis, makes the system stable. 

The easiest way to add a zero to the system is through the ESR of the capacitor. You need a capacitor anyways, so you are killing two birds with one stone here. 

The ESR matters, because it controls the placement of the zero. It should be low enough so that you can get the -20db/dec when you cross the 0db axis but low enough that the gain is below 0 dB before the next pole (usually due to the parastics).