How to give a user the rights to "clear cache"
You may try UI Cache Clear.

Project page says:


  
  Clears specific cache bin without flushing the whole site cache.
  Allows to clear cache without full administrator permissions.
  Provides setting to clear current page cache automatically when    clearing Block, Views or Panels cache.
  Admin menu and Shortcut integration