title: Elasticsearch bulk update is extremely slow
body: <p>I am indexing a large amount of daily data ~160GB per index into elasticsearch. I am facing this case where I need to update almost all the docs in the indices with a small amount of data(~16GB) which is of the format</p>
  
  <pre><code>id1,data1
  id1,data2
  id2,data1
  id2,data2
  id2,data3
  .
  .
  .
  </code></pre>
  
  <p>My update operations start happening at 16000 lines per second and in over 5 minutes it comes down to 1000 lines per second and doesnt go up after that. The update process for this 16GB of data is currently longer than the time it takes for my entire indexing of 160GB to happen</p>
  
  <p>My conf file for the update operation currently looks as follows </p>
  
  <pre><code>output
  {
      elasticsearch {
          action =&gt; "update"
          doc_as_upsert =&gt; true
          hosts =&gt; ["host1","host2","host3","host4"]
          index =&gt; "logstash-2017-08-1"
          document_id =&gt; "%{uniqueid}"
          document_type =&gt; "daily"
          retry_on_conflict =&gt; 2
          flush_size =&gt; 1000
      }
  
  }
  </code></pre>
  
  <p>The optimizations I have done to speed up indexing in my cluster based on the suggestions here <a href="https://www.elastic.co/guide/en/elasticsearch/guide/current/indexing-performance.html" rel="nofollow noreferrer">https://www.elastic.co/guide/en/elasticsearch/guide/current/indexing-performance.html</a> are</p>
  
  <ol>
  <li>Setting "indices.store.throttle.type" : "none"</li>
  <li>Index "refresh_interval" : "-1"</li>
  </ol>
  
  <p>I am running my cluster on 4 instances of the d2.8xlarge EC2 instances. I have allocated 30GB of heap to each nodes. 
  While the update is happening barely any cpu is used and the load is very less as well.</p>
  
  <p>Despite everything the update is extremely slow. Is there something very obvious that I am missing that is causing this issue? While looking at the threadpool data I find that the number of threads working on bulk operations are constantly high.</p>
  
  <p>Any help on this issue would be really helpful</p>
  
  <p>Thanks in advance</p>