Avoiding a call to CommandBarControl.Picture
SetButtonImage is called for every single CommandMenuItem in Rubberduck, regardless of whether or not Picture and Mask are overridden.

Whether a CommandBarButton has a Picture property depends on the [version of the] host application, and the bool result of HasPictureProperty will be the same for every single CommandMenuItem being initialized: if a RuntimeBinderException is thrown with the first button, it will be thrown and logged for every single button - this is a massive performance hit: throwing and catching exceptions like this, is pretty expensive.

I'm ok with the dynamic cast - though some reflection might actually perform better, but without changing the logic to stop making that test for every single button it would still be inefficient.

The first HasPictureProperty return value will be the value returned by every single subsequent call: you need to change the logic so as to avoid making that check more than once.