I have an issue with memory management in various operating systems.

My program is a server that does some processing that could take a few GB of memory. After that, it releases most of the memory while it waits for a few hours until another request arrives.

On AIX and Solaris, I observe the following behavior,

When I free memory, the memory is not returned back to the operating system. The amount of virtual memory used by a process always increases - never decreases. The same is true for the physical memory, up to its limit. Thus it appears that we use all this memory in sleep mode as well.

When this memory can be returned back to OS? How can I make it?

Linux is different: it appears that is does return memory sometimes, but I'm not able to understand when and how. I have for example a scenario in which the process before a request was 100MB, then 700MB at the peak, and after releasing all that it was down to 600MB. I don't understand it - if Linux gives back memory to the OS, why not all of it?