Machine code: How can an OS prevent user-level programs from accessing each other's memory?
The key is Virtual Memory.

There might be many ways to do this, but the reigning one across most OSes (*nix or Windowses alike) is Virtual Memory. And it needs CPU support, which is given in x86 by the MMU (Memory Management Unit) and in which I'll base my examples further on.

The secret is: no user-level process has access to the whole CPU. x86 has four rings of execution, and certain CPU opcodes (instructions) execute only within the appropriate ring. Ring 0 is for the kernel (the heart of the OS), and ring 3 is for user-level processes (rings 1 and 2 are most often not used, and if used, it's usually for virtualisation purposes -- out of the scope of this answer). If you execute an instruction within the wrong ring, the CPU faults to the kernel. Hence, if a process tries to execute a privilege instruction, when the CPU is on ring 3 mode, the CPU itself stops that process and gives control to the kernel, which will usually kill that process. Only the kernel can change the ring mode.

The secret-key for memory protection is that, all memory accesses, unless executed within the kernel's ring 0 (and perhaps even within that ring mode, but that's not relevant for this discussion, as we're talking only about user processes, not the kernel), need to go through the MMU, which will translate the memory address of the process to that which the kernel has actually given.

So two different processes can access a memory cell by the same address, say, 0x45b3c1, whatever. In reality, the MMU will translate real-time each process's request to what the kernel says it should be.

For this, the kernel prepares page tables, which you can imagine as a "map&lt;void*, void*&gt;" (forgive my over-simplification here, and try to imagine the crazy optimisations needed), of what the process actually has. And it prepares tables only for what the process previously said it would use. If the process asks for an address that hasn't previously asked the kernel to prepare a table for him, then the CPU says this process got nuts and in reference of this process, it signals the infamous SEGFAULT to the kernel, which will decide what to do next.

I know that this is sort of self-promotion, but I just recently wrote a lot about this topic and even gave a talk at my company, in case the link is welcomed I'd like to share: https://www.farfromready.com/malloc-alone-a-story-of-the-computers-memory/