Rigid body simulation of helical gears/oblique pressure vectors
I don't have the technical knowledge to explain why, but the Bullet physics library seems to like denser meshes.

Subdividing the tooth surfaces 4 times (equivalent to 15 cuts) made these gears work.