what is preventing me from changing difficulty every submit when pool mining?
The work you were given is associated with a difficulty that the server decided upon before giving you that work.

Any work results you send in must be associated with work data known by the server.

When you submit work results, the work is looked up, and voila, there is the difficulty.

You may be given new work that has a different difficulty associated with it at a later time, but the old work still has the old difficulty.