What specifically prevents OTAs from being taken on modified /systems, and why can't it be shut off? I have had a constant nightmare with not being able to take OTAs on any of my (rooted) devices. They boot into TWRP, complain about the build fingerprint, and won't install, so I have to wait for a full .zip and flash my systems manually.

What is it that is preventing the OTAs from installing over a modified /system partition, and why can't it be defeated or disabled?

If it can't be defeated/disabled, why can't my modified /system lie about its build fingerprint to fool this validation into allowing the OTAs to be installed?
OTA files work by patching files rather than replacing them will a full copy of the new version of the file. This means that it has to verify that the existing files are exactly as it expects or the patching process won't work (or could cause the file to become corrupt). If you were to fake the fingerprint and force the OTA to apply you might end up with a device that can't boot due to some files being corrupt.

Android 5.0 (i think) switched from verifying just the files it was patching to verifying the partition as a whole so any modification (even to a file that is not being patched) will cause this to fail.