CKEditor: *During* WYSIWYG edit, how to show caption beneath image?
As of CKEditor 4.3, they now have widgets in the CKEditor side. They have introduced the Image2 plugin/widget, which supports image captions. The problem I've seen here is that Drupal has not caught up with this, so if you're using the media module for a file browser, it looks like that functionality is not yet ported to this newer version. However, if you just care about captions, it works. 

See: http://ckeditor.com/addon/image2 for the plugin. I've been using it with the CKEditor Module, but it should work just as fine with WYSIWYG + Ckeditor Library.