How to know which process is causing high memory usage on DB?
The software you use is out of date. An advantage of that is that there could very well be a solution provided in My Oracle Support. The Oracle OEM agent have lot's of problems in many versions. Prepare for patching the agents or use a daily restart agent procedure. The Oracle OEM agent is part of the Grid Control monitoring infra structure so you could take it down without harming the application that is using the database[s] that the agent is monitoring.