How to Control Core Performance Cache Expire Time Of Day
Partial answer, but I typically handle exact scheduling and cron-but-not-cron tasks by using a custom script and calling it with wget in a crontab.  You can use cron.php for guidance on how to bootstrap, and then write your own function to clear what you want, then put it in crontab to run when you want.  Just remember to lock down the script in .htaccess or apache config.