How can I configure the backend to get a comfortable image insert functionality with captions?
Have you tried the Media module? From the project page: 


Add media (images and video) into any textarea (requires Wysiwyg module).
Use the fields API (formerly CCK) to add fields to any type of media, eg. captions on images.
Plugins to use remote media from sites like YouTube, Flickr, ...
A unified interface for adding any type of media as a field to nodes.
Mass importing of files.
Drag and drop upload (with Plupload module).
Responsive image support (with Picture module).