Cache field / entity / node
I currently am caching the result using cache_set()/cache_get() inside the hook_field_formatter_view function for my field


Honestly this is not a bad way to go about doing it. When you have custom logic which is resource intensive, taking responsibility for your own caching decisions is usually the right idea. Honestly, I really wish more developers would familiarize themselves with cache_get/set().

Entitycache appears to be an effort to do this generally, and over time I am sure that some best practices will emerge as more people work in this space. However as of today, for anyone looking at a similar problem-set: "Figure out how to cache it yourself" is a good answer.