Op-Amp based power supply, what powers the Op-Amps?
"Op-Amps need power supply and that also would have to be a stable one." Not really. The typical/cheap linear regulator (e.g. LM317) conceptually looks like the diagram below (taken from Clayton and Winder's Operational Amplifiers so it's somewhat authoritative.) The error-magnifying opamp (inside) is simply powered by the unregulated supply/input. The output of this opamp will be affected by variations in its (unregulated) supply in proportion to its PSRR. Nevertheless the output regulation it provides is quite good.



You can find actual LM317 circuits (down to transistor level) on the net, but don't ask me to analyse those here.