Opamp on a single power supply
You're very close. 

First, as you suspected, get rid of the diode.

Second, change R3 to 9.5k, or R4 to 10.5k. Your 2.5 volt reference will have a Thevenin equivalent resistance of 500 ohms, so you need to compensate in order to keep the gain right.

A few (hopefully minor) points. 

Your 5 volts must be stable and noise-free. Any drift or noise will cause drift or noise on the output.

It's probably not important, but be aware that the LM324 will not actually pull all the way to ground. You might expect a worst-case "ground saturation" of about 20 mV.

I'm not sure what your specific question means. The op amp will act to keep the voltage between the + and - inputs as close to each other as possible, within 2 or 3 mV (which is the offset voltage).