Using Media, views, and colorbox to create a image gallery
I don't see anything in bold but is the issue showing the image in your custom style or having it open in a modal?

For the latter, opening the image in a modal, you would need to set one field item to be the colorbox trigger.

The trigger needs to be a field you have already added, in your case the image field which you will use the custom display format with. Then you add the image field again with the desired popup format. You set this field to display within the colorbox trigger.

There are several related screencasts on drupal.org that may help:

https://drupal.org/project/colorbox