Wildcard Loader function called multiple times (_load)
There's an issue for this. It looks like you have to implement caching in your loader.

I was curious and tried it out on admin/structure/types/manage/%node_type. I grepped for node_type_load and nothing seemed to be calling it directly, so I put a ddebug_backtrace() in the load function. I got 13 traces when visiting that path! 10 of these came from menu_local_tasks() which loops through the local tasks and calls _menu_translate() which calls _menu_load_objects() for each router item. The other two extra calls came via menu_set_active_trail() which also called _menu_translate(). So I guess it depends on your menu structure how many times a loader might be called. As far as I can tell the object's loaded only to check whether or not it can be (access is denied if it can't). I guess this just reinforces Damien's comment about implementing static caching in your loader.