Permission to edit activities but not contacts?
The National Urban League funded work to add ACLs to cases, which laid the groundwork for adding ACLs to activities.  See here: https://civicrm.stackexchange.com/a/8005/12.  So one option would be to develop (or fund the development) of applying ACLs to activities.  I'm guessing the core team would be willing to take that on as a Paid Issue Queue item, since they did the groundwork.  This would be the most flexible arrangement.

Additionally, CiviDesk has implemented a hook that allows extension authors to escalate permissions via an extension with a new hook, hook_civicrm_permission_check.  This code has been written but at this time has not yet merged into upstream CiviCRM.  See CRM-19256 to see if that's still true now.  If you're able to QA this patch, please update the Jira ticket to say so.

The second approach is much less work to get into everyone's hands, but the first approach would allow non-coders to define their own activity permissions.