Why do some of my IE converter circuits have a large offset voltage?
Couple of theories here:


How sure are your power supplies are coming up symmetrically?
If one rail comes up before the other, you may have non-zero output voltages from the op-amp for a very short period of time.
Have you implemented all the PCB-layout practices needed for such high-impedances? At minimum, you're going to need guard-rings on all the ultra-high-impedance nodes.
The National (Now TI) LMC6082 datasheet has a good discussion of what is required to get board-leakage currents low enough to not be a problem.




This will likely not address the possibility that you have dielectric soakage issues, as discussed in @RocketSurgeon's answer.
A good and easy way to test his answer would be to desolder one of the caps on a bad board, and reverse it. If the offset is flipped in the other direction, it's a dielectric soakage issue (because the persistant charge in the cap will have a single polarity). If the offset voltage does not change, the problem is not the capacitor.

One thing I don't see the dielectric soakage issue explaining is why the charge seems to come back when the circuit is unpowered, and go away when it is powered. Since the element that discharges the capacitor is connected continuously across the cap, (e.g C1||R2, C2||R1), the contribution of any current leaking out of the cap should be a constant, and not affected by the supply voltage.

The only thing that comes to mind for me would be that there is something hygroscopic somewhere, and it injects an offset current. When you power the board, it warms up, and drives the moisture out over time. Turn the board off, and it begins to resorb moisture.



One comment I do have is I don't see why you have both SW1A, and SW1B. You can entirely dispose of SW1B. Just tie both of the R/C pairs together, and to the output of the op-amp. When one of the cap/resistor sets is selected, the other will just slowly discharge. As long as one end is floating (which is accomplished by SW1A), the voltage on the other end is irrelevant.