IK rigged elbows have unexpected "laggy" or "jumpy" behaviour when animated
So by messing around I managed to find my own mistake, and hopefully this answer will help others avoid it.

As is the standard with most IK setups, I have 2 bones in the place of the Hand:  


The actual deforming Hand bone, which is child of the Lower Arm bone
A helper "Controller" bone, which occupies the same space as the Hand bone, but is not a child of the skeleton.


The Lower arm bone has a regular IK with the hand Contoller as a target, the deforming Hand bone then has a Copy Rotation constraint, so that it always has the same orientation as the hand Controller.

And here's where I made my mistake:


I wanted to add some extra realism to my rig by adding a Copy Rotation constraint on top of the IK to the Lower arm, 0.5 weight and limited to Y, so that the forearm would twist slightly as the hand bone turns. But I've set the target of this constraint to be the deforming Hand bone, and not the Controller bone.
That means the IK solver had to compute a lower arm bone, which rotation was dependent on the hand bone, which rotation was dependent on the controller bone. The rotation of the Lower arm basically had to be calculated in two steps, which is most likely what caused the strange Elbow desync as seen in that gif above.


I've discovered this by the elbows suddenly fixing themselves when I muted the lower arm's copy rotation constraint.

So the solution is following:


As ironical as it is compared to the length of my descriptions, the solution was as simple as changing the target of the Lower arm's Copy Rotation constraint from the Hand bone to the hand Controller bone. That way the rotation for the lower arm is only calculated once, and the problem disappears.