Why is Blender's IK pole target system so difficult? Am I missing something?
You are trying to do impossible. Check this out how Pole target angle is exactly computed and how the IK works with it.

You either setup the pole angle or the IK base bone roll.

Adjusting the pole angle is lesser evil, that's how it should be done. Adjusting bone roll ends up in re-binding skin.

Maybe the iTaSC IK solver would suit better your application than the blender's legacy one, try it out.