how to get view_mode from within hook_field_formatter_view
Imho a formatter shouldn't be dependent on the view mode. It's the site builder's choice which formatter to use in which view mode, and through that decide how things look. It sounds to me like you'd be better off either having two formatters, or having one formatter which can be configured.

My understanding of the Drupal way here is that you provide the general purpose tools (just like a module typically provides permissions, but not a new role - leaving it to the site builder to aggregate perms into roles appropriate for their particular needs). Every time I create a new view mode I want to have complete power to make it do exactly what I want.