Where are my UV coordinates!
I discovered the problem!

I needed to to undo the tangle of my shader nodes and leave the normal map as the only thing connected to the normal node of the principled shader. Then it stopped freaking out.

So if you have this problem temporarily rearrange your shader nodes to have exclusively the textures you're working on connected.

I answer my own questions a lot I'm realizing.