Noninverting-amplifier offset
The TL08x series of opamps have a typical input voltage offset error on the order of 3 mV (datasheet here). This offset is indistinguishable from a real input, and it gets amplified the same way.

I assume you're talking about U1 in your diagram, which is configured for a gain of about 5000&times;. This means that the output error could be on the order of 15 V (!), so if you're measuring just a few volts, you're actually doing much better than "typical" in your setup.

Since you're already bandpassing your signal, you should include a low-frequency rolloff in your noninverting amplifier in order to minimize the effects of its input offset error. One common way to do that is to incorporate a "DC offset servo" (try googling that phrase) into the feedback of that stage.