Add contextual link to panels "custom content" pane?
Try FAPE, Field API Pane Editor. It is a separate module from ctools.

It does make it much easier for non-technical users to edit a field or a single pane of a page.