Where does Android store the binaries of the apps themselves?
Just answering as the comments suggested,

 /data/app/package.name/base.apk


The APK File is placed inside the data partition.

This is different from Windows as you mentioned.

Windows uses a different protection for System Software called UAC (User Access Control)
whereas Android uses a separated Partition on the HDD ( SSD ) called the System Partition.

All Stock Software is stored under 

system/app/*app name*/app name.apk


&amp;

system/priv-app/*app name*/app name.apk


or

system/app/appname.apk


&amp;

system/priv-app/appname.apk


Additional Software from OEM's is usually stored under

system/vendor/app/


However since the new partitions in Android 7.0+ i believe that Vendor is now designated its own partition, making the app directory

vendor/app


Or similar.

Some Vendors put APK's into the system tp be automata installed also under "custom" partition or root folder

custom/app


Furthermore, applications can be built by binaries under 

system/bin/


&amp;

system/xbin/


If you're the Administrator of your PC, you can hypothetically destroy the systems integrity on Windows - however due to any changes being placed in /data partition on Android, you can not delete system software or destroy the system integrity without root access.

The data partition &amp; cache partition hold every single modification made to the device - this means a "Factory Reset" just Erases the Data &amp; Cache partitions on Android - VS windows reinstalling the entire system from a separate backup partition.