title: Elasticsearch OutOfMemoryError Java heap space
body: <p>I am running an 8 cores, 32g RAM elasticsearch node with 5 shards, 400 million (small) documents.</p>
  
  <p>Everything works great until I run an agg search, then shards start failing with: </p>
  
  <pre><code>java.lang.OutOfMemoryError: Java heap space
  </code></pre>
  
  <p>I have <a href="https://www.elastic.co/guide/en/elasticsearch/guide/current/heap-sizing.html" rel="noreferrer">changed heap size</a> with:
  <code>export ES_HEAP_SIZE=16g</code> (also <code>ES_MAX_MEM</code> and <code>ES_MIN_MEM</code> to same) </p>
  
  <p>also changed the <code>yml</code> file for elasticsearch:</p>
  
  <pre><code>bootstrap.mlockall: true
  </code></pre>
  
  <p>and even (recommended by install documents):</p>
  
  <pre><code>sudo sysctl -w vm.max_map_count=262144
  </code></pre>
  
  <p>Restart service and still no no impact, still <code>java.lang.OutOfMemoryError: Java heap space</code></p>
  
  <p>Any other suggestions? other than don't run agg queries?</p>
  
  <p>query is:  </p>
  
  <pre><code>https://localhost:9200/my_index_name/_search?search_type=count
  {
    "aggs": {
      "distinct_hostname": {
        "cardinality": {
          "field": "hostname"
        }
      }
    }
  }
  </code></pre>