Using capacitor with lower voltage than the specified one
You are correct.  Generally speaking, capacitors must not be subjected to voltages higher than what they are specified for.  In practice, one always chooses a capacitor with voltage rating somewhat in excess of the highest voltage the capacitor might be exposed to.  For example, I would choose a 63V capacitor for a circuit running at 45V.

Your circuit is running at plus minus 20V.  A capacitor voltage rating of 25V is marginal, so I would choose a capacitor voltage rating at 35V or more.  Your 50V capacitor will be just fine.