How can we provide a stable reference voltage to a voltage regulator if we don't have a stable source to begin with?
The primary reference is something like a zener diode or a band-gap reference or maybe a floating gate MOSFET. The regulator is something like a power amplifier that provides an output that is proportional to that voltage. The gain of that amplifier determines how much the output voltage changes with load current changes (load regulation). 

Although the reference can operate okay from unregulated input, there will be some change with input voltage (line regulation). To improve that regulation we can start up the reference in some way and run it mostly from a voltage or current related to the output voltage. This is called "bootstrapping".  

Here is the block diagram of an MC78M05 showing the basic regulator and protection circuitry. 



Here is how the schematic of an LM7805 breaks down: 



The green part is the startup circuitry. The yellow is the bandgap reference. The rest is the error amplifier, feedback divider and protection circuitry.