Angular resolution of NXT motor
I found the resolution of the EV3 motors by attaching a 90° connection to the motor, and a long axle with a pointer at the end so I could see changes in small angles.  By running a loop a large number of times with settings of 360°, then with 360.5°, then 360.75°, and watching for when the changes trigger actual accumulation of ending point, I found that there was no difference between a rotation of 360° and 360.75°, but at 361° the difference appeared.

I'll try the same setup with an NXT motor and let you know the results.

So it's like a step function with an interval of 1°.

I have building instructions and the program that I'll but on Rebrickable sometime this week.

Hope this helps.