Objects tracked to camera track to the viewport camera at current frame instead of camera location at the frame it should be rendering
I've found a solution to get the text to show up correctly with relatively little effort. By going to correct frame, and using Object &gt; Rigid Body &gt; Apply Transformation like so:

After doing that I could remove the problematic constraints but keep most of the effect I wanted, it did mean that for one shot the text doesn't follow the camera, but at least it wasn't completely reversed anymore, so I counted that as a win.
Since this doesn't actually solve the constraints not updating properly whilst rendering the animation I won't accept this answer, but as a temporary fix I thought it worth mentioning.