Correct Capacitors to use with L7805 Voltage Regulator
The answers are in the notes:


  
  Although no output capacitor is need for stability, it does improve transient response.
  Required if regulator is locate an appreciable distance from power supply filter.
  


So  the values aren't critical to the operation, but you should have them there. Bigger capacitors give more energy to the regulator and more energy to your circuit (input and output capacitors respectively) but at the cost of lower frequency response. The actual values you select depend on your circuit, not the regulator. 10µF is a good "ballpark" figure to use in most circumstances.

The input capacitor reduces the impedance of the power source by providing a local source of power. The output capacitor is like the "reservoir" capacitor you should have at the power entry to all boards anyway, and reduces the impedance to the rest of your circuit.

For the higher frequency responses you should have smaller value decoupling capacitors by all your active components anyway.

You can also add more smaller capacitors to the output of the regulator to improve high frequency responsiveness if needed - you're not just limited to the one capacitor.