title: ElasticSearch 150k query takes several minutes
body: <p>I am trying to query roughly 150k docs from ES. My 150k docs sum up to 9.24GB in disk space and I am using the scroll API and fetch 10'000 per scroll. It takes well beyond 5min to query it and runs into memory problems (requires more than 6GB of RAM). Now I tried several approaches:</p>
  
  <p>_source: regular filtering. From the 70 key-value pairs per document I only need 4 fields</p>
  
  <pre><code> {
    "size": 10000,
    "query": {
      "range": {
        "updateTime": {
          "lte": 10000000000000000
        }
      }
    },
    "_source": [
      "id.identifierOne",
      "id.identifierTwo",
      "arrivalTime",
      "updateTime"
    ],
    "sort": [
      "_doc"
    ]
  }
  </code></pre>
  
  <p>_stored_field: in order to omit the huge overhead of the full doc, I store 4 fields during index creation</p>
  
  <pre><code>{
    "size": 100000,
    "stored_fields": [
      "updateTime",
      "id.identifierOne",
      "id.identifierTwo",
      "arrivalTime"
    ],
    "query": {
      "range": {
        "updateTime": {
          "lte": 10000000000000000
        }
      }
    },
    "sort": [
      "_doc"
    ]
  }
  </code></pre>
  
  <p>Both run in equal time. The other weird thing I noticed with the _source version is, every 10'000 scroll leads to longer query times for the next batch and increasing RAM.</p>
  
  <p>I strongly suspect an error in my config or query. Waiting for several minutes seems to be way too long.</p>
  
  <p>My config is: 5 shards on one node only. The pod scales to 2CPU's with 6GB of RAM. Index creation is done like this</p>
  
  <pre><code> {
        "mappings": {
          "doc": {
            "dynamic": "false",
            "properties": {
              "id.identifierOne": {
                "type": "long",
                "store": true
              },
              "id.identifierTwo": {
                "type": "text",
                "store": true
              },
              "arrivalTime": {
                "type": "long",
                "store": true
              },
              "updateTime": {
                "type": "long",
                "store": true
              }
            }
          },
    "settings" : {
      "index" : {
        "number_of_replicas" : 0
      }
    }
  }
  </code></pre>
  
  <p>Is there a possibility to reduce the query time to under a minute?</p>
  
  <p>Any help or ideas are highly appreciated.</p>