Sharp sensor not working with 9 V Battery
Have you tried to measure the sensor supply voltage with an oscilloscope to see if there are any noise spikes or voltage sags present?  

The sharp distance sensor works by sending out short infrared pulses, that's why it will also draw current in pulses. The datasheet specifies the supply current with 30mA, but that is just the average over time. The instantaneous current draw during a pulse can be much higher.  

Your 9V battery has quite a high internal resistance, which could lead to the voltage after the 7805 dropping below the minimum supply voltage during a pulse. That's why you have to add a large decoupling capacitor as close as possible to the sensor, which the datasheet recommends anyway on page 6:  


  In order to stabilize power supply line, we recommend to insert a by-pass capacitor of 10uF or more between Vcc and GND near this product


If you really want to be sure, you can solder a 10uF and 0.1uF capacitor directly to the sensor:  


  

The reason it works with the Arduino board is probably because it already has enough bulk capacitance on its 5V supply rail. The schematic for the Arduino Uno shows a 47uF capacitor.