Different objects selected on different tabs
Maybe it won't make the process as fast as you'd like, but if you disable the Lock Object Modes option it will speed up a bit when you'll switch from the armature to the object and back to the armature.