Very low frequency Passive High Pass filter. Capacitor type
First, ceramic capacitors are readily available to 10s of &micro;F at the low voltages you are using.  Ceramic capacitors can be non-linear when the applied voltage gets near the end of its specified range.  You therefore want to get one specified significantly more than the offset you want to remove.  It seems you want to remove less than 2 V of DC bias, so even "10 V" ceramic caps should do quite well.

Second, if you want a more accurate high pass rolloff frequency, then err on the low side, and do the real high pass in firmware.  Maybe you don't even need to do a high pass at all in hardware.  This depends on the A/D range and how the signal fits within it.  If the signal will always be within the A/D range with enough resolution, then this is a firmware problem, not a hardware problem.