Is there a reason why with symmetric padding, the inverse wavelet transform is not the adjoint of the wavelet transform?
Actually the redundancy perfectly explains why the adjoint property is not verified. The reconstruction operator is not orthogonal anymore and therefore the inverse is not necessarily its adjoint.

You can compute the adjoint efficiently according to https://arxiv.org/pdf/1707.02018.pdf.

Regarding the necessity of redundance, according to this person it is theoretically needed, although he couldn't provided a resource or a proof of that.
But since it's a separate issue, I am going to ask another question (you can check it here).