When you change the skill used, do any modifiers for the original skill still count?
Modifiers related to the old skill are discarded and don't count, unless they are Mission bonuses/penalties. rules


  Skill Substitution: Some special abilities and cards allow the ninja to substitute one skill for another. In this case, the roll is against the new skill. If the Mission card specified bonuses or penalties to the roll, they still apply to the new skill. However, cards relating to the old skill can no longer be used (and if they had already been used, they are discarded without effect). For instance, “Little Kids Watching” affects Disguise skill. If you sub- stitute another skill for Disguise, then “Little Kids Watching” can no longer affect that roll.


In your first example, the new skill is Climbing not combat, so your opponents -3 Combat modifier doesn't apply and is discarded. In your second example, the mission is specifying a -2 Combat modifier, its -2 Mission penalty continues to apply to the new skill.