High resistance op amp
Look at the LM324 specification. The bias current is typically 20nA, and in your circuit that comes from your source. With a perfect opamp you would have 4nA flowing through your 1Gohm (total) potential divider, so the bias current upsets that somewhat. Do you really need such a high impedance source?

Such high impedances can lead to problems, particularly from leakage currents. Furthermore, if you must use this circuit configuration, you need an opamp with a maximum bias current of around 40pA - such as TL084 - and great care with cleanliness. My recommendation is to reduce the source impedance.