Drupal 7 - How to get AJAX response displayed in a ctools modal?
Checkout Documentation on: http://api.drupal.org/api/drupal/includes!ajax.inc/7
or: http://drupal.org/node/1635198#comment-6120546 on how to render drupal messages.

For more Information about ctools modal you may check: Example to display webforms (webform.module) within CTools modal windows 

Or what about a tutorial like: "Ajax modal windows, the easy way"