Will an electrolytic capacitor work reliably with 0 VDC and 100 mVAC?
Yes, it appears that this is OK for such small voltages.

According to CDE's Application Guide, Aluminum Electrolytic Capacitors:


  Aluminum electrolytic capacitors are polarized and must be
  connected in the correct polarity. They can withstand
  reverse voltages up to 1.5 V.


If you are uncomfortable with that (from the same application guide):


  If two, same-value, aluminum electrolytic capacitors
  are connected in series, back-to-back with the positive
  terminals or the negative terminals connected, the
  resulting single capacitor is a non-polar capacitor with
  half the capacitance.


For solid Tantalum capacitors, from Vishay's Solid Tantalum Capacitors:
Frequently Asked Questions (FAQs):


  Tantalum capacitors are capable of withstanding peak voltages in the reverse direction equal to 10 % of the DC rating at
  + 25 °C and 5 % of the DC rating at + 85 °C.


Also, you may not want to consider a ceramic capacitor for this due to the piezoelectric effects: Ceramic vs. Film Capacitor: Which one is preferred in audio circuits?