title: kubenrates (k8s) nd POD OOMKILLED during not working hours and no memory spikes
body: <p>This is the situation,<br />
  first time on AWS, first time on K8s...<br />
  we have a microservice infrastructure,<br />
  we have memory issue with pods/container with jdk8 so we move to jdk11-openj9
  memory management if far way well than before, the issue now is a strange one,
  a microservice, that is working fine with the assigned memory, get OOMKILL during the not working hours, at random hours, today ar 5.03 AM
  this POD is configured to have a QOS = GUARANTEED and this is the only one so EVICTION is not what is happening.<br />
  We monitored (with grafana) also the other PODS in that NODE and at the killing time, no one has a memory spike</p>
  <p>bitly (the one killed): [scrennshot from grafana]<img src="https://i.stack.imgur.com/o980D.png" alt="https://i.imgur.com/zphWYkr.png" /></p>
  <p>config: [scrennshot from grafana]<img src="https://i.imgur.com/XUubMoy.png" alt="https://i.imgur.com/XUubMoy.png" /></p>
  <p>oauth2: [scrennshot from grafana]<img src="https://i.imgur.com/AsupFGI.png" alt="https://i.imgur.com/AsupFGI.png" /></p>
  <p>redis: [scrennshot from grafana]<img src="https://i.imgur.com/VN4wBPm.png" alt="https://i.imgur.com/VN4wBPm.png" /></p>
  <p>customactivity: [scrennshot from grafana]<img src="https://i.imgur.com/q8ztYx8.png" alt="https://i.imgur.com/q8ztYx8.png" /></p>
  <p>chat: [scrennshot from grafana]<img src="https://i.imgur.com/4Hp6FaV.png" alt="https://i.imgur.com/4Hp6FaV.png" /></p>
  <p>someone have a suggestion where to look?</p>
  <p>OOMKILLED sure not for memory excessive request in my opinion.</p>
  <p>update 15/05/2020</p>
  <p>killed again this morning :
  State:          Running
  Started:      Fri, 15 May 2020 01:11:27 +0000
  Last State:     Terminated
  Reason:       OOMKilled
  Exit Code:    137
  Started:      Wed, 13 May 2020 07:32:46 +0000
  Finished:     Fri, 15 May 2020 01:11:27 +0000
  Ready:          True
  Restart Count:  1
  Limits:
  memory:  250Mi
  Requests:
  cpu:        100m
  memory:     250Mi</p>
  <p>no reason why OOMKILLED, no memory peak or system overload</p>