Armature animation is transformed incorrectly and cannot reset to its default position
I figured it out. When I was animating I accidentally got into edit mode and pressed ALT-R (thought that I was in pose mode), which reset the roll position of the bones.
Recalculating the roll position brings the bones to the rest position when no transforms are applied.

However, the animation still has weird behavior, it might be that the recalculated roll is different to what I had when rigging the bones.
Anyway, here's my tip: RECALCULATE ROLL BEFORE ANIMATING. So if you accidentally changed the roll, you only have to recalculate the roll and not to reanimate all the animations.