What do the Software Update options do?
You're correct.

I have "Install App updates" and "Install system data files and security updates" checked in System Preferences and "Install OS X updates" deselected, and exactly what you describe is what I observe.

OS X Updates are user-facing updates that directly affect the user, regardless of whether the user actually uses the respective app (e.g. RAW updates).
The system/security updates are the Security Updates and File Quarantine (XProtect) definitions.

In the last 30 days according to my App Store Updates tab, the following was not auto-installed for me with "Install OS X updates" disabled:


Command Line Tools (OS X 10.10) 6.3
Digital Camera RAW Compatibility Update 6.04
Remote Desktop Client Update 3.8.4


OS X 10.11.0 would be considered a separate app on the App Store, so it wouldn't be automatically updated from a version prior to 10.11.0 regardless of the settings.