I run a medium VPS (4GB RAM) as a web server. I host some private Ruby on Rails sites with Apache and mod_passenger. I also have two legacy apps which run with passenger standalone (two Nginx servers with passenger on ports 3000 and 4000). Apache acts as a proxy for those.Most of the time this is works okay, but every few months the whole system goes berserk: RAM usage goes up to 100%the log files get filled with Cannot allocate memory & reverse proxy errorssoon enough the whole system is unresponsiveSometimes this clears up by itself, sometimes I have to ask my provider to reboot the system because I can't even SSH into it.This always happens at night, when traffic is very low. Here are two graphs:Apache accessesMemory consumptionHow do I get to the bottom of this? Which steps can I take to debug this situation? Is it possible that I have a bad neighbor on the machine?Other processes on the major machine: MySQL, PostgreSQL, Elasticsearch, RedisUPDATE to clarifyI'm looking for a tool that records processes with their memory & cpu consumption so I can track down the trouble maker(s). Is something like that out there or do I have to roll my own?
Weird memory spikes in Debian VPS
linux;debian;apache httpd;memory;nginx
From my personal experience of managing RoR apps, memory leaks can easily creep in and can catch you at most unexpected time. I would recommend doing two things to catch the culprit:Use sar to record the system activities. Since memory is specific target, use free | grep Mem: | awk '{print $4}' and send out a mail to yourself using mailx if the threshold exceeds a certain limit of your choice.Usually there is a bad sql query or a rouge gem that cause this. If anything has recently changed with respect to these, then you may want to look into that as well.