title: Kubernetes throwing OOM for pods running a JVM
body: <p>I am running Docker containers containing JVM (java8u31). These containers are deployed as pods in a kubernetes cluster. Often I get OOM for the pods and Kubernetes kills the pods and restarts it. I am having issues in finding the root cause for these OOMs as I am new to Kubernetes.</p>
  
  <ol>
  <li><p>Here are the JVM parameters</p>
  
  <pre><code>-XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -Xms700M -Xmx1000M  -XX:MaxRAM=1536M  -XX:MaxMetaspaceSize=250M 
  </code></pre></li>
  <li><p>These containers are deployed as stateful set and following is the resource allocation</p>
  
  <pre><code>resources:
      requests:
          memory: "1.5G"
          cpu: 1
      limits:
          memory: "1.5G"
          cpu: 1
  </code></pre>
  
  <p>so the total memory allocated to the container matches the MaxRam </p></li>
  <li><p>If I use <code>-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/etc/opt/jmx/java_pid%p.hprof</code> that doesn't help because the pod is getting killed and recreated and started as soon as there is a OOM so everything within the pod is lost</p>
  
  <p>The only way to get a thread or HEAP dump is to SSH into the pod which also I am not able to take because the pod is recreated after an OOM so I don't get the memory footprint at the time of OOM. I SSH after an OOM which is not much help.</p></li>
  <li><p>I also profiled the code using visualVM, jHat but couldn't find substantial memory footprint which could lead to a conclusion of too much memory consumption by the threads running within the JVM or a probable leak.</p></li>
  </ol>
  
  <p>Any help is appreciated to resolve the OOM thrown by Kubernetes.</p>