LM358 Operational amplifier Output voltage rising with stable inputs
It could be input offset voltage drift with temperature. 

The data sheet says it is typically 7 uV/degC and with a gain about 680 (10k resistor set to zero), this will cause a shift in output voltage of 4.7 mV for every degC that the LM358 warms up. It should eventually settle down of course.

Try touching the LM358 with the tip of a soldering iron to see if there are significant rapid shifts in output voltage.