RFI on the amplifier due to Wifi
I would say the most likely culprit for a DC shift is the photo-diode itself. Depending on the lighting condition it will develop a bias. With indoor lighting 50Hz/60Hz flicker is likely, and there can be plenty of RF there too. Also the wifi-module probably has diodes (possibly blinking rapidly) which could be another source of noise. Try shielding the diode from light to see how that affects the situation. If it is the problem then consider changing the input connection or topology. You could bias the diode with a resistor and send the signal via a coupling capacitor, then you can adjust R and C to filter low-frequency components.