Send PDF email through CiviRules
I ended up hardcoding it so it uses $contact['email'] instead of $params['to_email']

I am leaving it open for some time, if anybody has a better idea (or if I missed something obvious)

UPDATE:

The developers of the extension included this feature. Using version 2.0 the PDF will be emailed to the associated contact if the to email field is left empty. So no need for hacks anymore. Big thanks to ErikH - CiviCooP