Creating thank you letters redirects to a contribution page
There didn't seem to be anything related in the CiviCRM log (with logging turned on) or PHP error log. In the Apache error log there was a strange error:

AH01797: client denied by server configuration: 
/home/username/public_html/sites/default/files/civicrm/custom/delete-this-MDH295Ww25


Permissions all seemed fine, however.

I ended up installing wkhtmltopdf and bypassing dompdf which isn't really a solution to the problem but is an alternative.