How can I let authenticated users rebuild caches?
Yes, that's what I created Rebuild Cache Access for.


  This module provides a new permission: rebuild cache access. Every
  role with this permission set will have a nice and big "Rebuild Cache"
  admin toolbar button. Clicking this button simply fires
  drupal_flush_all_caches(). That's all.