How does the OS handle multiple stacks in its physical memory?
The point you make at the very end of your question is the right one.
No modern computer has a stack. In hardware terms it turns out to be superfluous, wasteful and useless.
A stack is superfluous because one can easily perform the operations “write a value”, “retrieve the last value written”, and so on by having a pointer to a memory location. “Push” can be implemented by advancing the pointer and writing to the location it points to; “pop” by reading the pointed-to location and undoing the advance of the pointer. (On Intel architectures the pointer is a register called SP, and “advancing” decrements SP before writing to memory). This emulation of a stack requires no hardware at all - only memory, which exists already.
A hardware stack is wasteful because it requires special mechanisms over above the “read at location”, “write at location” which we already have because we have RAM. (Some early or embedded microprocessor architectures have gone the other way, having a very few levels of stack on-chip).
A hardware stack is useless because fashionable computer languages such as C cannot function with a stack in the strict sense - they need to access not just the last pushed value (such as a return address) but values pushed earlier on, which can be the arguments to a subroutine, for example.
Returning therefore to your main question: among the memory assigned to a process there is an area of memory which the process will use for its stack. Its stack pointer will point into that area. Switching from one process to another involves saving the whole state of the old process including the stack pointer and loading the whole state of the new process including the stack pointer. Because there is no special stack as such, on process termination everything is simply forgotten: nothing else needs to be done.