<p dir="auto">It calls <code class="notranslate">assertDuelOptimization</code> which is fairly confusing but doesn't seem to assert a whole lot. In fact I'm not really sure what it does assert.</p>
<p dir="auto">When you run the whole suite with <code class="notranslate">@Seed("A72FE66BDD9F236C")</code> it GCs fairly hard.</p>
<p dir="auto">This cam up in:<br>
<a href="https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-os-compatibility/os=ubuntu/726/console" rel="nofollow">https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-os-compatibility/os=ubuntu/726/console</a></p>
<p dir="auto">with this see:<br>
<code class="notranslate">gradle :core:integTest -Dtests.seed=9D639ADBB7706439 -Dtests.class=org.elasticsearch.search.sort.GeoDistanceIT -Dtests.method="testDuelOptimizations" -Dtests.security.manager=true -Dtests.locale=el -Dtests.timezone=CET</code></p>
<p dir="auto">I run into:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="==&gt; Test Info: seed=9D639ADBB7706439; jvm=1; suite=1
Suite: org.elasticsearch.search.sort.GeoDistanceIT
  1&gt; [2016-07-05 10:38:56,227][WARN ][org.elasticsearch.bootstrap] Unable to lock JVM Memory: error=78, reason=Function not implemented
  1&gt; [2016-07-05 10:38:56,229][WARN ][org.elasticsearch.bootstrap] This can result in part of the JVM being swapped out.
  1&gt; [2016-07-05 10:38:59,333][INFO ][org.elasticsearch.search.sort] [GeoDistanceIT#testDuelOptimizations]: setup test
  1&gt; [2016-07-05 10:38:59,404][INFO ][org.elasticsearch.test   ] Setup InternalTestCluster [SUITE-CHILD_VM=[0]-CLUSTER_SEED=[5060446888734553576]-HASH=[1046378D01417]-cluster] with seed [463A51A247C0F5E8] using [0] dedicated masters, [3] (data) nodes and [0] coord only nodes
  1&gt; [2016-07-05 10:39:00,853][INFO ][org.elasticsearch.node   ] [node_s0] version[5.0.0-alpha4-SNAPSHOT], pid[27139], build[Unknown/Unknown], OS[Mac OS X/10.11.5/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_66/25.66-b17]
  1&gt; [2016-07-05 10:39:00,853][INFO ][org.elasticsearch.node   ] [node_s0] initializing ...
  1&gt; [2016-07-05 10:39:00,894][INFO ][org.elasticsearch.plugins] [node_s0] modules [], plugins [org.elasticsearch.test.InternalSettingsPlugin, org.elasticsearch.test.ESIntegTestCase$TestSeedPlugin]
  1&gt; [2016-07-05 10:39:05,899][INFO ][org.elasticsearch.env    ] [node_s0] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [100.8gb], net total_space [232.6gb], spins? [unknown], types [hfs]
  1&gt; [2016-07-05 10:39:05,899][INFO ][org.elasticsearch.env    ] [node_s0] heap size [491mb], compressed ordinary object pointers [true]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:39:09, stalled for 10.2s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:39:11,629][INFO ][org.elasticsearch.node   ] [node_s0] initialized
  1&gt; [2016-07-05 10:39:11,631][INFO ][org.elasticsearch.node   ] [node_s0] starting ...
  1&gt; [2016-07-05 10:39:11,637][INFO ][org.elasticsearch.transport] [node_s0] publish_address {local[1]}, bound_addresses {local[1]}
  1&gt; [2016-07-05 10:39:11,683][INFO ][org.elasticsearch.cluster.service] [node_s0] new_master {node_s0}{HeZ3nsHmTeGJV-xBA8Jnqw}{0z-S-nOMTZCUCGx8mo96tw}{local}{local[1]}, reason: local-disco-initial_connect(master)
  1&gt; [2016-07-05 10:39:11,693][INFO ][org.elasticsearch.node   ] [node_s0] started
  1&gt; [2016-07-05 10:39:11,702][INFO ][org.elasticsearch.node   ] [node_s1] version[5.0.0-alpha4-SNAPSHOT], pid[27139], build[Unknown/Unknown], OS[Mac OS X/10.11.5/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_66/25.66-b17]
  1&gt; [2016-07-05 10:39:11,702][INFO ][org.elasticsearch.node   ] [node_s1] initializing ...
  1&gt; [2016-07-05 10:39:11,702][INFO ][org.elasticsearch.plugins] [node_s1] modules [], plugins [org.elasticsearch.test.InternalSettingsPlugin, org.elasticsearch.test.ESIntegTestCase$TestSeedPlugin]
  1&gt; [2016-07-05 10:39:11,721][INFO ][org.elasticsearch.env    ] [node_s1] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [100.8gb], net total_space [232.6gb], spins? [unknown], types [hfs]
  1&gt; [2016-07-05 10:39:11,721][INFO ][org.elasticsearch.env    ] [node_s1] heap size [491mb], compressed ordinary object pointers [true]
  1&gt; [2016-07-05 10:39:11,818][INFO ][org.elasticsearch.gateway] [node_s0] recovered [0] indices into cluster_state
  1&gt; [2016-07-05 10:39:11,984][INFO ][org.elasticsearch.node   ] [node_s1] initialized
  1&gt; [2016-07-05 10:39:11,985][INFO ][org.elasticsearch.node   ] [node_s1] starting ...
  1&gt; [2016-07-05 10:39:11,991][INFO ][org.elasticsearch.transport] [node_s1] publish_address {local[2]}, bound_addresses {local[2]}
  1&gt; [2016-07-05 10:39:11,994][INFO ][org.elasticsearch.cluster.service] [node_s0] added {{node_s1}{BAEudZyYTtKvHdV3mCitGA}{A5mMsb-jTqS5Ks27ZBM2Ng}{local}{local[2]},}, reason: local-disco-receive(from node[{node_s1}{BAEudZyYTtKvHdV3mCitGA}{A5mMsb-jTqS5Ks27ZBM2Ng}{local}{local[2]}])
  1&gt; [2016-07-05 10:39:12,006][INFO ][org.elasticsearch.cluster.service] [node_s1] detected_master {node_s0}{HeZ3nsHmTeGJV-xBA8Jnqw}{0z-S-nOMTZCUCGx8mo96tw}{local}{local[1]}, added {{node_s0}{HeZ3nsHmTeGJV-xBA8Jnqw}{0z-S-nOMTZCUCGx8mo96tw}{local}{local[1]},}, reason: local-disco-receive(from master)
  1&gt; [2016-07-05 10:39:12,016][INFO ][org.elasticsearch.node   ] [node_s1] started
  1&gt; [2016-07-05 10:39:12,031][INFO ][org.elasticsearch.node   ] [node_s2] version[5.0.0-alpha4-SNAPSHOT], pid[27139], build[Unknown/Unknown], OS[Mac OS X/10.11.5/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_66/25.66-b17]
  1&gt; [2016-07-05 10:39:12,031][INFO ][org.elasticsearch.node   ] [node_s2] initializing ...
  1&gt; [2016-07-05 10:39:12,032][INFO ][org.elasticsearch.plugins] [node_s2] modules [], plugins [org.elasticsearch.test.InternalSettingsPlugin, org.elasticsearch.test.ESIntegTestCase$TestSeedPlugin]
  1&gt; [2016-07-05 10:39:12,045][INFO ][org.elasticsearch.env    ] [node_s2] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [100.8gb], net total_space [232.6gb], spins? [unknown], types [hfs]
  1&gt; [2016-07-05 10:39:12,045][INFO ][org.elasticsearch.env    ] [node_s2] heap size [491mb], compressed ordinary object pointers [true]
  1&gt; [2016-07-05 10:39:12,110][INFO ][org.elasticsearch.node   ] [node_s2] initialized
  1&gt; [2016-07-05 10:39:12,110][INFO ][org.elasticsearch.node   ] [node_s2] starting ...
  1&gt; [2016-07-05 10:39:12,111][INFO ][org.elasticsearch.transport] [node_s2] publish_address {local[3]}, bound_addresses {local[3]}
  1&gt; [2016-07-05 10:39:12,113][INFO ][org.elasticsearch.cluster.service] [node_s0] added {{node_s2}{kEqhOhmdRSSF_ZMEtNcjiA}{IkTxKRhtQ8KTk-9Cyr3GPw}{local}{local[3]},}, reason: local-disco-receive(from node[{node_s2}{kEqhOhmdRSSF_ZMEtNcjiA}{IkTxKRhtQ8KTk-9Cyr3GPw}{local}{local[3]}])
  1&gt; [2016-07-05 10:39:12,136][INFO ][org.elasticsearch.cluster.service] [node_s1] added {{node_s2}{kEqhOhmdRSSF_ZMEtNcjiA}{IkTxKRhtQ8KTk-9Cyr3GPw}{local}{local[3]},}, reason: local-disco-receive(from master)
  1&gt; [2016-07-05 10:39:12,138][INFO ][org.elasticsearch.cluster.service] [node_s2] detected_master {node_s0}{HeZ3nsHmTeGJV-xBA8Jnqw}{0z-S-nOMTZCUCGx8mo96tw}{local}{local[1]}, added {{node_s0}{HeZ3nsHmTeGJV-xBA8Jnqw}{0z-S-nOMTZCUCGx8mo96tw}{local}{local[1]},{node_s1}{BAEudZyYTtKvHdV3mCitGA}{A5mMsb-jTqS5Ks27ZBM2Ng}{local}{local[2]},}, reason: local-disco-receive(from master)
  1&gt; [2016-07-05 10:39:12,145][INFO ][org.elasticsearch.node   ] [node_s2] started
  1&gt; [2016-07-05 10:39:12,198][INFO ][org.elasticsearch.search.sort] test using _default_ mappings: [{&quot;_default_&quot;:{}}]
  1&gt; [2016-07-05 10:39:12,449][INFO ][org.elasticsearch.search.sort] [GeoDistanceIT#testDuelOptimizations]: starting test
  1&gt; [2016-07-05 10:39:12,555][INFO ][org.elasticsearch.cluster.metadata] [node_s0] [index] creating index, cause [api], templates [random_index_template], shards [7]/[1], mappings [_default_, type]
  1&gt; [2016-07-05 10:39:12,758][INFO ][org.elasticsearch.plugins] [transport_client_node_s1] modules [], plugins []
  1&gt; [2016-07-05 10:39:12,791][INFO ][org.elasticsearch.transport] [transport_client_node_s1] publish_address {local[4]}, bound_addresses {local[4]}
  1&gt; [2016-07-05 10:39:12,804][INFO ][org.elasticsearch.plugins] [transport_client_node_s0] modules [], plugins []
  1&gt; [2016-07-05 10:39:12,823][INFO ][org.elasticsearch.transport] [transport_client_node_s0] publish_address {local[5]}, bound_addresses {local[5]}
  1&gt; [2016-07-05 10:39:12,885][INFO ][org.elasticsearch.plugins] [transport_client_node_s2] modules [], plugins []
  1&gt; [2016-07-05 10:39:12,907][INFO ][org.elasticsearch.transport] [transport_client_node_s2] publish_address {local[6]}, bound_addresses {local[6]}
  1&gt; [2016-07-05 10:39:13,030][INFO ][org.elasticsearch.cluster.routing.allocation] [node_s0] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[index][1], [index][2], [index][5], [index][4], [index][2], [index][5]] ...]).
  1&gt; [2016-07-05 10:39:13,264][INFO ][org.elasticsearch.cluster.routing.allocation] [node_s0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[index][5], [index][2], [index][2], [index][5]] ...]).
  1&gt; [2016-07-05 10:39:13,645][INFO ][org.elasticsearch.search.sort] Index [10195] docs async: [false] bulk: [true] partitions [11]
  1&gt; [2016-07-05 10:39:13,753][INFO ][org.elasticsearch.cluster.metadata] [node_s0] [index/mJJm22qWT_S83ojJ_oIxUQ] create_mapping [RANDOM_BOGUS_TYPE______]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:39:19, stalled for 20.2s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:39:25,297][INFO ][org.elasticsearch.search.sort] Now testing GeoDistance=ARC, distance=1797.0km, origin=(-59.52571451701207, -97.76032857491252)
  1&gt; [2016-07-05 10:39:25,645][INFO ][org.elasticsearch.search.sort] 121 hits
  1&gt; [2016-07-05 10:39:25,645][INFO ][org.elasticsearch.search.sort] Now testing GeoDistance=SLOPPY_ARC, distance=1797.0km, origin=(-59.52571451701207, -97.76032857491252)
  1&gt; [2016-07-05 10:39:25,679][INFO ][org.elasticsearch.search.sort] 121 hits
  1&gt; [2016-07-05 10:39:25,679][INFO ][org.elasticsearch.search.sort] Now testing GeoDistance=ARC, distance=8402.0km, origin=(-30.631741764098656, -172.2723431422603)
  1&gt; [2016-07-05 10:39:26,265][INFO ][org.elasticsearch.search.sort] 1921 hits
  1&gt; [2016-07-05 10:39:26,265][INFO ][org.elasticsearch.search.sort] Now testing GeoDistance=SLOPPY_ARC, distance=8402.0km, origin=(-30.631741764098656, -172.2723431422603)
  1&gt; [2016-07-05 10:39:26,527][INFO ][org.elasticsearch.search.sort] 1921 hits
  1&gt; [2016-07-05 10:39:26,528][INFO ][org.elasticsearch.search.sort] Now testing GeoDistance=ARC, distance=1363.0km, origin=(-69.94611459615871, -151.37323663888245)
  1&gt; [2016-07-05 10:39:26,584][INFO ][org.elasticsearch.search.sort] 103 hits
  1&gt; [2016-07-05 10:39:26,584][INFO ][org.elasticsearch.search.sort] Now testing GeoDistance=SLOPPY_ARC, distance=1363.0km, origin=(-69.94611459615871, -151.37323663888245)
  1&gt; [2016-07-05 10:39:26,616][INFO ][org.elasticsearch.search.sort] 103 hits
  1&gt; [2016-07-05 10:39:26,616][INFO ][org.elasticsearch.search.sort] Now testing GeoDistance=ARC, distance=1317.0km, origin=(-86.05583303929738, 112.60762123704916)
  1&gt; [2016-07-05 10:39:28,774][INFO ][org.elasticsearch.monitor.jvm] [node_s0] [gc][17] overhead, spent [335ms] collecting in the last [1s]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:39:29, stalled for 30.2s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:39:30,074][INFO ][org.elasticsearch.monitor.jvm] [node_s1] [gc][18] overhead, spent [316ms] collecting in the last [1s]
  1&gt; [2016-07-05 10:39:30,173][INFO ][org.elasticsearch.monitor.jvm] [node_s2] [gc][18] overhead, spent [316ms] collecting in the last [1s]
  1&gt; [2016-07-05 10:39:34,018][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][19] overhead, spent [3.7s] collecting in the last [3.9s]
  1&gt; [2016-07-05 10:39:34,018][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][19] overhead, spent [3.7s] collecting in the last [3.8s]
  1&gt; [2016-07-05 10:39:34,037][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][19] overhead, spent [3.8s] collecting in the last [4.2s]
  1&gt; [2016-07-05 10:39:36,673][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][20] overhead, spent [2.2s] collecting in the last [2.6s]
  1&gt; [2016-07-05 10:39:36,673][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][20] overhead, spent [2.2s] collecting in the last [2.6s]
  1&gt; [2016-07-05 10:39:36,673][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][20] overhead, spent [2.2s] collecting in the last [2.6s]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:39:39, stalled for 40.2s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:39:49, stalled for 50.2s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:39:59, stalled for 60.2s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:40:09, stalled for 70.3s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:40:19, stalled for 80.3s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:40:29, stalled for 90.3s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:40:39, stalled for  100s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:40:49, stalled for  110s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:40:59, stalled for  120s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:41:09, stalled for  130s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:41:19, stalled for  140s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:41:29, stalled for  150s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:41:39, stalled for  160s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:41:47,363][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][21] overhead, spent [1.9m] collecting in the last [9.2s]
  1&gt; [2016-07-05 10:41:47,365][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][21] overhead, spent [48.2s] collecting in the last [2.6s]
  1&gt; [2016-07-05 10:41:47,376][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][21] overhead, spent [48.2s] collecting in the last [2.6s]
  1&gt; [2016-07-05 10:41:47,428][WARN ][org.elasticsearch.transport] [transport_client_node_s0] Received response for a request that has timed out, sent [96692ms] ago, timed out [41ms] ago, action [cluster:monitor/nodes/liveness], node [{#transport#-1}{65cGS8JbRICTfQiJ2J0NXg}{local}{local[1]}], id [63]
  1&gt; [2016-07-05 10:41:47,429][INFO ][org.elasticsearch.client.transport] [transport_client_node_s0] failed to get node info for {#transport#-1}{65cGS8JbRICTfQiJ2J0NXg}{local}{local[1]}, disconnecting...
  1&gt; ReceiveTimeoutTransportException[[][local[1]][cluster:monitor/nodes/liveness] request_id [63] timed out after [96651ms]]
  1&gt;    at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:820)
  1&gt;    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:391)
  1&gt;    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  1&gt;    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  1&gt;    at java.lang.Thread.run(Thread.java:745)
  1&gt; [2016-07-05 10:41:47,429][INFO ][org.elasticsearch.client.transport] [transport_client_node_s1] failed to get node info for {#transport#-1}{yzWaILXVRrK2bShnxGu_gQ}{local}{local[2]}, disconnecting...
  1&gt; ReceiveTimeoutTransportException[[][local[2]][cluster:monitor/nodes/liveness] request_id [65] timed out after [83184ms]]
  1&gt;    at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:820)
  1&gt;    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:391)
  1&gt;    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  1&gt;    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  1&gt;    at java.lang.Thread.run(Thread.java:745)
  1&gt; [2016-07-05 10:41:47,428][WARN ][org.elasticsearch.transport] [transport_client_node_s1] Received response for a request that has timed out, sent [91531ms] ago, timed out [8347ms] ago, action [cluster:monitor/nodes/liveness], node [{#transport#-1}{yzWaILXVRrK2bShnxGu_gQ}{local}{local[2]}], id [65]
  2&gt; Ιουλ 05, 2016 10:41:47 ΠΜ com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
  2&gt; WARNING: Uncaught exception in thread: Thread[elasticsearch[node_s1][search][T#2],5,TGRP-GeoDistanceIT]
  2&gt; java.lang.OutOfMemoryError: GC overhead limit exceeded
  2&gt;    at __randomizedtesting.SeedInfo.seed([9D639ADBB7706439]:0)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:98)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:71)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:71)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:71)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:71)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:71)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  1&gt; [2016-07-05 10:41:48,371][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][22] overhead, spent [1.1m] collecting in the last [2.1m]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:41:49, stalled for  170s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:41:48,382][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][22] overhead, spent [1.1m] collecting in the last [2.1m]
  1&gt; [2016-07-05 10:41:49,711][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][23] overhead, spent [690ms] collecting in the last [1.3s]
  1&gt; [2016-07-05 10:41:49,711][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][23] overhead, spent [690ms] collecting in the last [1.3s]
  1&gt; [2016-07-05 10:41:49,712][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][23] overhead, spent [690ms] collecting in the last [1.3s]
  1&gt; [2016-07-05 10:41:50,806][INFO ][org.elasticsearch.monitor.jvm] [node_s1] [gc][24] overhead, spent [341ms] collecting in the last [1s]
  1&gt; [2016-07-05 10:41:50,806][INFO ][org.elasticsearch.monitor.jvm] [node_s0] [gc][24] overhead, spent [341ms] collecting in the last [1s]
  1&gt; [2016-07-05 10:41:50,806][INFO ][org.elasticsearch.monitor.jvm] [node_s2] [gc][24] overhead, spent [341ms] collecting in the last [1s]
  1&gt; [2016-07-05 10:41:58,032][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][27] overhead, spent [5s] collecting in the last [5.2s]
  1&gt; [2016-07-05 10:41:58,033][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][27] overhead, spent [5s] collecting in the last [5.2s]
  1&gt; [2016-07-05 10:41:58,032][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][27] overhead, spent [5s] collecting in the last [5.2s]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:42:00, stalled for  182s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:42:00,687][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][28] overhead, spent [2.3s] collecting in the last [2.6s]
  1&gt; [2016-07-05 10:42:00,687][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][28] overhead, spent [2.3s] collecting in the last [2.6s]
  1&gt; [2016-07-05 10:42:00,687][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][28] overhead, spent [2.3s] collecting in the last [2.6s]
  1&gt; [2016-07-05 10:42:02,523][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][29] overhead, spent [1.6s] collecting in the last [1.8s]
  1&gt; [2016-07-05 10:42:02,523][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][29] overhead, spent [1.6s] collecting in the last [1.8s]
  1&gt; [2016-07-05 10:42:02,523][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][29] overhead, spent [1.6s] collecting in the last [1.8s]
  1&gt; [2016-07-05 10:42:03,911][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][30] overhead, spent [1.1s] collecting in the last [1.3s]
  1&gt; [2016-07-05 10:42:03,911][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][30] overhead, spent [1.1s] collecting in the last [1.3s]
  1&gt; [2016-07-05 10:42:03,912][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][30] overhead, spent [1.1s] collecting in the last [1.3s]
  1&gt; [2016-07-05 10:42:05,680][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][31] overhead, spent [1.3s] collecting in the last [1.7s]
  1&gt; [2016-07-05 10:42:05,680][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][31] overhead, spent [1.3s] collecting in the last [1.7s]
  1&gt; [2016-07-05 10:42:05,681][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][31] overhead, spent [1.3s] collecting in the last [1.7s]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:42:10, stalled for  192s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:42:12,722][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][33] overhead, spent [4.9s] collecting in the last [5.9s]
  1&gt; [2016-07-05 10:42:12,722][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][33] overhead, spent [4.9s] collecting in the last [5.9s]
  1&gt; [2016-07-05 10:42:12,722][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][33] overhead, spent [4.9s] collecting in the last [5.9s]
  1&gt; [2016-07-05 10:42:15,004][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][34] overhead, spent [1.7s] collecting in the last [2.2s]
  1&gt; [2016-07-05 10:42:15,004][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][34] overhead, spent [1.7s] collecting in the last [2.2s]
  1&gt; [2016-07-05 10:42:15,004][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][34] overhead, spent [1.7s] collecting in the last [2.2s]
  1&gt; [2016-07-05 10:42:16,731][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][35] overhead, spent [1.3s] collecting in the last [1.7s]
  1&gt; [2016-07-05 10:42:16,731][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][35] overhead, spent [1.3s] collecting in the last [1.7s]
  1&gt; [2016-07-05 10:42:16,731][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][35] overhead, spent [1.3s] collecting in the last [1.7s]
  1&gt; [2016-07-05 10:42:19,183][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][36] overhead, spent [2.2s] collecting in the last [2.4s]
  1&gt; [2016-07-05 10:42:19,183][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][36] overhead, spent [2.2s] collecting in the last [2.4s]
  1&gt; [2016-07-05 10:42:19,184][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][36] overhead, spent [2.2s] collecting in the last [2.4s]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:42:20, stalled for  202s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:42:22,605][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][38] overhead, spent [2.1s] collecting in the last [2.4s]
  1&gt; [2016-07-05 10:42:22,605][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][38] overhead, spent [2.1s] collecting in the last [2.4s]
  1&gt; [2016-07-05 10:42:22,605][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][38] overhead, spent [2.1s] collecting in the last [2.4s]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:42:30, stalled for  212s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:42:32,959][INFO ][org.elasticsearch.monitor.jvm] [node_s2] [gc][old][41][64] duration [7.4s], collections [1]/[8.3s], total [7.4s]/[2.5m], memory [249.4mb]-&gt;[301.3mb]/[491mb], all_pools {[young] [5mb]-&gt;[189.6kb]/[57.5mb]}{[survivor] [56.5mb]-&gt;[0b]/[56.5mb]}{[old] [187.8mb]-&gt;[301.1mb]/[341.5mb]}
  1&gt; [2016-07-05 10:42:32,959][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][41] overhead, spent [7.7s] collecting in the last [8.3s]
  1&gt; [2016-07-05 10:42:32,960][INFO ][org.elasticsearch.monitor.jvm] [node_s1] [gc][old][41][64] duration [7.4s], collections [1]/[8.3s], total [7.4s]/[2.5m], memory [249.4mb]-&gt;[301.3mb]/[491mb], all_pools {[young] [5mb]-&gt;[225kb]/[57.5mb]}{[survivor] [56.5mb]-&gt;[0b]/[56.5mb]}{[old] [187.8mb]-&gt;[301.1mb]/[341.5mb]}
  1&gt; [2016-07-05 10:42:32,960][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][41] overhead, spent [7.7s] collecting in the last [8.3s]
  1&gt; [2016-07-05 10:42:32,960][INFO ][org.elasticsearch.monitor.jvm] [node_s0] [gc][old][41][64] duration [7.4s], collections [1]/[8.3s], total [7.4s]/[2.5m], memory [249.4mb]-&gt;[301.6mb]/[491mb], all_pools {[young] [5mb]-&gt;[464.7kb]/[57.5mb]}{[survivor] [56.5mb]-&gt;[0b]/[56.5mb]}{[old] [187.8mb]-&gt;[301.1mb]/[341.5mb]}
  1&gt; [2016-07-05 10:42:32,960][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][41] overhead, spent [7.7s] collecting in the last [8.3s]
  1&gt; [2016-07-05 10:42:36,837][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][42] overhead, spent [3.1s] collecting in the last [3.8s]
  1&gt; [2016-07-05 10:42:36,837][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][42] overhead, spent [3.1s] collecting in the last [3.8s]
  1&gt; [2016-07-05 10:42:36,837][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][42] overhead, spent [3.1s] collecting in the last [3.8s]
  1&gt; [2016-07-05 10:42:38,746][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][43] overhead, spent [1.5s] collecting in the last [1.9s]
  1&gt; [2016-07-05 10:42:38,746][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][43] overhead, spent [1.5s] collecting in the last [1.9s]
  1&gt; [2016-07-05 10:42:38,746][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][43] overhead, spent [1.5s] collecting in the last [1.9s]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:42:40, stalled for  222s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:42:50, stalled for  232s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:43:00, stalled for  242s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:43:10, stalled for  252s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:43:20, stalled for  262s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:43:30, stalled for  272s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:43:40, stalled for  282s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:43:50, stalled for  292s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:44:00, stalled for  302s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:44:10, stalled for  312s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:44:21, stalled for  322s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:44:31, stalled for  332s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:44:41, stalled for  342s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:44:51, stalled for  352s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:45:01, stalled for  362s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:45:11, stalled for  372s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:45:21, stalled for  382s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:45:31, stalled for  392s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:45:41, stalled for  402s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:45:51, stalled for  412s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:46:01, stalled for  422s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:46:11, stalled for  432s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:46:21, stalled for  442s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:46:31, stalled for  452s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:46:41, stalled for  462s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:46:51, stalled for  472s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:47:01, stalled for  482s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:47:11, stalled for  492s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:47:21, stalled for  502s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:47:31, stalled for  512s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:47:41, stalled for  522s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:47:51, stalled for  532s at: GeoDistanceIT.testDuelOptimizations"><pre class="notranslate"><code class="notranslate">==&gt; Test Info: seed=9D639ADBB7706439; jvm=1; suite=1
Suite: org.elasticsearch.search.sort.GeoDistanceIT
  1&gt; [2016-07-05 10:38:56,227][WARN ][org.elasticsearch.bootstrap] Unable to lock JVM Memory: error=78, reason=Function not implemented
  1&gt; [2016-07-05 10:38:56,229][WARN ][org.elasticsearch.bootstrap] This can result in part of the JVM being swapped out.
  1&gt; [2016-07-05 10:38:59,333][INFO ][org.elasticsearch.search.sort] [GeoDistanceIT#testDuelOptimizations]: setup test
  1&gt; [2016-07-05 10:38:59,404][INFO ][org.elasticsearch.test   ] Setup InternalTestCluster [SUITE-CHILD_VM=[0]-CLUSTER_SEED=[5060446888734553576]-HASH=[1046378D01417]-cluster] with seed [463A51A247C0F5E8] using [0] dedicated masters, [3] (data) nodes and [0] coord only nodes
  1&gt; [2016-07-05 10:39:00,853][INFO ][org.elasticsearch.node   ] [node_s0] version[5.0.0-alpha4-SNAPSHOT], pid[27139], build[Unknown/Unknown], OS[Mac OS X/10.11.5/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_66/25.66-b17]
  1&gt; [2016-07-05 10:39:00,853][INFO ][org.elasticsearch.node   ] [node_s0] initializing ...
  1&gt; [2016-07-05 10:39:00,894][INFO ][org.elasticsearch.plugins] [node_s0] modules [], plugins [org.elasticsearch.test.InternalSettingsPlugin, org.elasticsearch.test.ESIntegTestCase$TestSeedPlugin]
  1&gt; [2016-07-05 10:39:05,899][INFO ][org.elasticsearch.env    ] [node_s0] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [100.8gb], net total_space [232.6gb], spins? [unknown], types [hfs]
  1&gt; [2016-07-05 10:39:05,899][INFO ][org.elasticsearch.env    ] [node_s0] heap size [491mb], compressed ordinary object pointers [true]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:39:09, stalled for 10.2s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:39:11,629][INFO ][org.elasticsearch.node   ] [node_s0] initialized
  1&gt; [2016-07-05 10:39:11,631][INFO ][org.elasticsearch.node   ] [node_s0] starting ...
  1&gt; [2016-07-05 10:39:11,637][INFO ][org.elasticsearch.transport] [node_s0] publish_address {local[1]}, bound_addresses {local[1]}
  1&gt; [2016-07-05 10:39:11,683][INFO ][org.elasticsearch.cluster.service] [node_s0] new_master {node_s0}{HeZ3nsHmTeGJV-xBA8Jnqw}{0z-S-nOMTZCUCGx8mo96tw}{local}{local[1]}, reason: local-disco-initial_connect(master)
  1&gt; [2016-07-05 10:39:11,693][INFO ][org.elasticsearch.node   ] [node_s0] started
  1&gt; [2016-07-05 10:39:11,702][INFO ][org.elasticsearch.node   ] [node_s1] version[5.0.0-alpha4-SNAPSHOT], pid[27139], build[Unknown/Unknown], OS[Mac OS X/10.11.5/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_66/25.66-b17]
  1&gt; [2016-07-05 10:39:11,702][INFO ][org.elasticsearch.node   ] [node_s1] initializing ...
  1&gt; [2016-07-05 10:39:11,702][INFO ][org.elasticsearch.plugins] [node_s1] modules [], plugins [org.elasticsearch.test.InternalSettingsPlugin, org.elasticsearch.test.ESIntegTestCase$TestSeedPlugin]
  1&gt; [2016-07-05 10:39:11,721][INFO ][org.elasticsearch.env    ] [node_s1] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [100.8gb], net total_space [232.6gb], spins? [unknown], types [hfs]
  1&gt; [2016-07-05 10:39:11,721][INFO ][org.elasticsearch.env    ] [node_s1] heap size [491mb], compressed ordinary object pointers [true]
  1&gt; [2016-07-05 10:39:11,818][INFO ][org.elasticsearch.gateway] [node_s0] recovered [0] indices into cluster_state
  1&gt; [2016-07-05 10:39:11,984][INFO ][org.elasticsearch.node   ] [node_s1] initialized
  1&gt; [2016-07-05 10:39:11,985][INFO ][org.elasticsearch.node   ] [node_s1] starting ...
  1&gt; [2016-07-05 10:39:11,991][INFO ][org.elasticsearch.transport] [node_s1] publish_address {local[2]}, bound_addresses {local[2]}
  1&gt; [2016-07-05 10:39:11,994][INFO ][org.elasticsearch.cluster.service] [node_s0] added {{node_s1}{BAEudZyYTtKvHdV3mCitGA}{A5mMsb-jTqS5Ks27ZBM2Ng}{local}{local[2]},}, reason: local-disco-receive(from node[{node_s1}{BAEudZyYTtKvHdV3mCitGA}{A5mMsb-jTqS5Ks27ZBM2Ng}{local}{local[2]}])
  1&gt; [2016-07-05 10:39:12,006][INFO ][org.elasticsearch.cluster.service] [node_s1] detected_master {node_s0}{HeZ3nsHmTeGJV-xBA8Jnqw}{0z-S-nOMTZCUCGx8mo96tw}{local}{local[1]}, added {{node_s0}{HeZ3nsHmTeGJV-xBA8Jnqw}{0z-S-nOMTZCUCGx8mo96tw}{local}{local[1]},}, reason: local-disco-receive(from master)
  1&gt; [2016-07-05 10:39:12,016][INFO ][org.elasticsearch.node   ] [node_s1] started
  1&gt; [2016-07-05 10:39:12,031][INFO ][org.elasticsearch.node   ] [node_s2] version[5.0.0-alpha4-SNAPSHOT], pid[27139], build[Unknown/Unknown], OS[Mac OS X/10.11.5/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_66/25.66-b17]
  1&gt; [2016-07-05 10:39:12,031][INFO ][org.elasticsearch.node   ] [node_s2] initializing ...
  1&gt; [2016-07-05 10:39:12,032][INFO ][org.elasticsearch.plugins] [node_s2] modules [], plugins [org.elasticsearch.test.InternalSettingsPlugin, org.elasticsearch.test.ESIntegTestCase$TestSeedPlugin]
  1&gt; [2016-07-05 10:39:12,045][INFO ][org.elasticsearch.env    ] [node_s2] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [100.8gb], net total_space [232.6gb], spins? [unknown], types [hfs]
  1&gt; [2016-07-05 10:39:12,045][INFO ][org.elasticsearch.env    ] [node_s2] heap size [491mb], compressed ordinary object pointers [true]
  1&gt; [2016-07-05 10:39:12,110][INFO ][org.elasticsearch.node   ] [node_s2] initialized
  1&gt; [2016-07-05 10:39:12,110][INFO ][org.elasticsearch.node   ] [node_s2] starting ...
  1&gt; [2016-07-05 10:39:12,111][INFO ][org.elasticsearch.transport] [node_s2] publish_address {local[3]}, bound_addresses {local[3]}
  1&gt; [2016-07-05 10:39:12,113][INFO ][org.elasticsearch.cluster.service] [node_s0] added {{node_s2}{kEqhOhmdRSSF_ZMEtNcjiA}{IkTxKRhtQ8KTk-9Cyr3GPw}{local}{local[3]},}, reason: local-disco-receive(from node[{node_s2}{kEqhOhmdRSSF_ZMEtNcjiA}{IkTxKRhtQ8KTk-9Cyr3GPw}{local}{local[3]}])
  1&gt; [2016-07-05 10:39:12,136][INFO ][org.elasticsearch.cluster.service] [node_s1] added {{node_s2}{kEqhOhmdRSSF_ZMEtNcjiA}{IkTxKRhtQ8KTk-9Cyr3GPw}{local}{local[3]},}, reason: local-disco-receive(from master)
  1&gt; [2016-07-05 10:39:12,138][INFO ][org.elasticsearch.cluster.service] [node_s2] detected_master {node_s0}{HeZ3nsHmTeGJV-xBA8Jnqw}{0z-S-nOMTZCUCGx8mo96tw}{local}{local[1]}, added {{node_s0}{HeZ3nsHmTeGJV-xBA8Jnqw}{0z-S-nOMTZCUCGx8mo96tw}{local}{local[1]},{node_s1}{BAEudZyYTtKvHdV3mCitGA}{A5mMsb-jTqS5Ks27ZBM2Ng}{local}{local[2]},}, reason: local-disco-receive(from master)
  1&gt; [2016-07-05 10:39:12,145][INFO ][org.elasticsearch.node   ] [node_s2] started
  1&gt; [2016-07-05 10:39:12,198][INFO ][org.elasticsearch.search.sort] test using _default_ mappings: [{"_default_":{}}]
  1&gt; [2016-07-05 10:39:12,449][INFO ][org.elasticsearch.search.sort] [GeoDistanceIT#testDuelOptimizations]: starting test
  1&gt; [2016-07-05 10:39:12,555][INFO ][org.elasticsearch.cluster.metadata] [node_s0] [index] creating index, cause [api], templates [random_index_template], shards [7]/[1], mappings [_default_, type]
  1&gt; [2016-07-05 10:39:12,758][INFO ][org.elasticsearch.plugins] [transport_client_node_s1] modules [], plugins []
  1&gt; [2016-07-05 10:39:12,791][INFO ][org.elasticsearch.transport] [transport_client_node_s1] publish_address {local[4]}, bound_addresses {local[4]}
  1&gt; [2016-07-05 10:39:12,804][INFO ][org.elasticsearch.plugins] [transport_client_node_s0] modules [], plugins []
  1&gt; [2016-07-05 10:39:12,823][INFO ][org.elasticsearch.transport] [transport_client_node_s0] publish_address {local[5]}, bound_addresses {local[5]}
  1&gt; [2016-07-05 10:39:12,885][INFO ][org.elasticsearch.plugins] [transport_client_node_s2] modules [], plugins []
  1&gt; [2016-07-05 10:39:12,907][INFO ][org.elasticsearch.transport] [transport_client_node_s2] publish_address {local[6]}, bound_addresses {local[6]}
  1&gt; [2016-07-05 10:39:13,030][INFO ][org.elasticsearch.cluster.routing.allocation] [node_s0] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[index][1], [index][2], [index][5], [index][4], [index][2], [index][5]] ...]).
  1&gt; [2016-07-05 10:39:13,264][INFO ][org.elasticsearch.cluster.routing.allocation] [node_s0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[index][5], [index][2], [index][2], [index][5]] ...]).
  1&gt; [2016-07-05 10:39:13,645][INFO ][org.elasticsearch.search.sort] Index [10195] docs async: [false] bulk: [true] partitions [11]
  1&gt; [2016-07-05 10:39:13,753][INFO ][org.elasticsearch.cluster.metadata] [node_s0] [index/mJJm22qWT_S83ojJ_oIxUQ] create_mapping [RANDOM_BOGUS_TYPE______]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:39:19, stalled for 20.2s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:39:25,297][INFO ][org.elasticsearch.search.sort] Now testing GeoDistance=ARC, distance=1797.0km, origin=(-59.52571451701207, -97.76032857491252)
  1&gt; [2016-07-05 10:39:25,645][INFO ][org.elasticsearch.search.sort] 121 hits
  1&gt; [2016-07-05 10:39:25,645][INFO ][org.elasticsearch.search.sort] Now testing GeoDistance=SLOPPY_ARC, distance=1797.0km, origin=(-59.52571451701207, -97.76032857491252)
  1&gt; [2016-07-05 10:39:25,679][INFO ][org.elasticsearch.search.sort] 121 hits
  1&gt; [2016-07-05 10:39:25,679][INFO ][org.elasticsearch.search.sort] Now testing GeoDistance=ARC, distance=8402.0km, origin=(-30.631741764098656, -172.2723431422603)
  1&gt; [2016-07-05 10:39:26,265][INFO ][org.elasticsearch.search.sort] 1921 hits
  1&gt; [2016-07-05 10:39:26,265][INFO ][org.elasticsearch.search.sort] Now testing GeoDistance=SLOPPY_ARC, distance=8402.0km, origin=(-30.631741764098656, -172.2723431422603)
  1&gt; [2016-07-05 10:39:26,527][INFO ][org.elasticsearch.search.sort] 1921 hits
  1&gt; [2016-07-05 10:39:26,528][INFO ][org.elasticsearch.search.sort] Now testing GeoDistance=ARC, distance=1363.0km, origin=(-69.94611459615871, -151.37323663888245)
  1&gt; [2016-07-05 10:39:26,584][INFO ][org.elasticsearch.search.sort] 103 hits
  1&gt; [2016-07-05 10:39:26,584][INFO ][org.elasticsearch.search.sort] Now testing GeoDistance=SLOPPY_ARC, distance=1363.0km, origin=(-69.94611459615871, -151.37323663888245)
  1&gt; [2016-07-05 10:39:26,616][INFO ][org.elasticsearch.search.sort] 103 hits
  1&gt; [2016-07-05 10:39:26,616][INFO ][org.elasticsearch.search.sort] Now testing GeoDistance=ARC, distance=1317.0km, origin=(-86.05583303929738, 112.60762123704916)
  1&gt; [2016-07-05 10:39:28,774][INFO ][org.elasticsearch.monitor.jvm] [node_s0] [gc][17] overhead, spent [335ms] collecting in the last [1s]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:39:29, stalled for 30.2s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:39:30,074][INFO ][org.elasticsearch.monitor.jvm] [node_s1] [gc][18] overhead, spent [316ms] collecting in the last [1s]
  1&gt; [2016-07-05 10:39:30,173][INFO ][org.elasticsearch.monitor.jvm] [node_s2] [gc][18] overhead, spent [316ms] collecting in the last [1s]
  1&gt; [2016-07-05 10:39:34,018][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][19] overhead, spent [3.7s] collecting in the last [3.9s]
  1&gt; [2016-07-05 10:39:34,018][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][19] overhead, spent [3.7s] collecting in the last [3.8s]
  1&gt; [2016-07-05 10:39:34,037][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][19] overhead, spent [3.8s] collecting in the last [4.2s]
  1&gt; [2016-07-05 10:39:36,673][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][20] overhead, spent [2.2s] collecting in the last [2.6s]
  1&gt; [2016-07-05 10:39:36,673][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][20] overhead, spent [2.2s] collecting in the last [2.6s]
  1&gt; [2016-07-05 10:39:36,673][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][20] overhead, spent [2.2s] collecting in the last [2.6s]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:39:39, stalled for 40.2s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:39:49, stalled for 50.2s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:39:59, stalled for 60.2s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:40:09, stalled for 70.3s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:40:19, stalled for 80.3s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:40:29, stalled for 90.3s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:40:39, stalled for  100s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:40:49, stalled for  110s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:40:59, stalled for  120s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:41:09, stalled for  130s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:41:19, stalled for  140s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:41:29, stalled for  150s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:41:39, stalled for  160s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:41:47,363][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][21] overhead, spent [1.9m] collecting in the last [9.2s]
  1&gt; [2016-07-05 10:41:47,365][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][21] overhead, spent [48.2s] collecting in the last [2.6s]
  1&gt; [2016-07-05 10:41:47,376][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][21] overhead, spent [48.2s] collecting in the last [2.6s]
  1&gt; [2016-07-05 10:41:47,428][WARN ][org.elasticsearch.transport] [transport_client_node_s0] Received response for a request that has timed out, sent [96692ms] ago, timed out [41ms] ago, action [cluster:monitor/nodes/liveness], node [{#transport#-1}{65cGS8JbRICTfQiJ2J0NXg}{local}{local[1]}], id [63]
  1&gt; [2016-07-05 10:41:47,429][INFO ][org.elasticsearch.client.transport] [transport_client_node_s0] failed to get node info for {#transport#-1}{65cGS8JbRICTfQiJ2J0NXg}{local}{local[1]}, disconnecting...
  1&gt; ReceiveTimeoutTransportException[[][local[1]][cluster:monitor/nodes/liveness] request_id [63] timed out after [96651ms]]
  1&gt;    at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:820)
  1&gt;    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:391)
  1&gt;    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  1&gt;    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  1&gt;    at java.lang.Thread.run(Thread.java:745)
  1&gt; [2016-07-05 10:41:47,429][INFO ][org.elasticsearch.client.transport] [transport_client_node_s1] failed to get node info for {#transport#-1}{yzWaILXVRrK2bShnxGu_gQ}{local}{local[2]}, disconnecting...
  1&gt; ReceiveTimeoutTransportException[[][local[2]][cluster:monitor/nodes/liveness] request_id [65] timed out after [83184ms]]
  1&gt;    at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:820)
  1&gt;    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:391)
  1&gt;    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  1&gt;    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  1&gt;    at java.lang.Thread.run(Thread.java:745)
  1&gt; [2016-07-05 10:41:47,428][WARN ][org.elasticsearch.transport] [transport_client_node_s1] Received response for a request that has timed out, sent [91531ms] ago, timed out [8347ms] ago, action [cluster:monitor/nodes/liveness], node [{#transport#-1}{yzWaILXVRrK2bShnxGu_gQ}{local}{local[2]}], id [65]
  2&gt; Ιουλ 05, 2016 10:41:47 ΠΜ com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
  2&gt; WARNING: Uncaught exception in thread: Thread[elasticsearch[node_s1][search][T#2],5,TGRP-GeoDistanceIT]
  2&gt; java.lang.OutOfMemoryError: GC overhead limit exceeded
  2&gt;    at __randomizedtesting.SeedInfo.seed([9D639ADBB7706439]:0)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:98)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:71)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:71)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:71)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:71)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:71)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.relateAndRecurse(GeoPointNumericTermsEnum.java:101)
  2&gt;    at org.apache.lucene.spatial.geopoint.search.GeoPointNumericTermsEnum.computeRange(GeoPointNumericTermsEnum.java:70)
  1&gt; [2016-07-05 10:41:48,371][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][22] overhead, spent [1.1m] collecting in the last [2.1m]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:41:49, stalled for  170s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:41:48,382][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][22] overhead, spent [1.1m] collecting in the last [2.1m]
  1&gt; [2016-07-05 10:41:49,711][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][23] overhead, spent [690ms] collecting in the last [1.3s]
  1&gt; [2016-07-05 10:41:49,711][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][23] overhead, spent [690ms] collecting in the last [1.3s]
  1&gt; [2016-07-05 10:41:49,712][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][23] overhead, spent [690ms] collecting in the last [1.3s]
  1&gt; [2016-07-05 10:41:50,806][INFO ][org.elasticsearch.monitor.jvm] [node_s1] [gc][24] overhead, spent [341ms] collecting in the last [1s]
  1&gt; [2016-07-05 10:41:50,806][INFO ][org.elasticsearch.monitor.jvm] [node_s0] [gc][24] overhead, spent [341ms] collecting in the last [1s]
  1&gt; [2016-07-05 10:41:50,806][INFO ][org.elasticsearch.monitor.jvm] [node_s2] [gc][24] overhead, spent [341ms] collecting in the last [1s]
  1&gt; [2016-07-05 10:41:58,032][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][27] overhead, spent [5s] collecting in the last [5.2s]
  1&gt; [2016-07-05 10:41:58,033][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][27] overhead, spent [5s] collecting in the last [5.2s]
  1&gt; [2016-07-05 10:41:58,032][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][27] overhead, spent [5s] collecting in the last [5.2s]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:42:00, stalled for  182s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:42:00,687][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][28] overhead, spent [2.3s] collecting in the last [2.6s]
  1&gt; [2016-07-05 10:42:00,687][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][28] overhead, spent [2.3s] collecting in the last [2.6s]
  1&gt; [2016-07-05 10:42:00,687][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][28] overhead, spent [2.3s] collecting in the last [2.6s]
  1&gt; [2016-07-05 10:42:02,523][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][29] overhead, spent [1.6s] collecting in the last [1.8s]
  1&gt; [2016-07-05 10:42:02,523][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][29] overhead, spent [1.6s] collecting in the last [1.8s]
  1&gt; [2016-07-05 10:42:02,523][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][29] overhead, spent [1.6s] collecting in the last [1.8s]
  1&gt; [2016-07-05 10:42:03,911][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][30] overhead, spent [1.1s] collecting in the last [1.3s]
  1&gt; [2016-07-05 10:42:03,911][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][30] overhead, spent [1.1s] collecting in the last [1.3s]
  1&gt; [2016-07-05 10:42:03,912][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][30] overhead, spent [1.1s] collecting in the last [1.3s]
  1&gt; [2016-07-05 10:42:05,680][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][31] overhead, spent [1.3s] collecting in the last [1.7s]
  1&gt; [2016-07-05 10:42:05,680][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][31] overhead, spent [1.3s] collecting in the last [1.7s]
  1&gt; [2016-07-05 10:42:05,681][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][31] overhead, spent [1.3s] collecting in the last [1.7s]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:42:10, stalled for  192s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:42:12,722][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][33] overhead, spent [4.9s] collecting in the last [5.9s]
  1&gt; [2016-07-05 10:42:12,722][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][33] overhead, spent [4.9s] collecting in the last [5.9s]
  1&gt; [2016-07-05 10:42:12,722][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][33] overhead, spent [4.9s] collecting in the last [5.9s]
  1&gt; [2016-07-05 10:42:15,004][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][34] overhead, spent [1.7s] collecting in the last [2.2s]
  1&gt; [2016-07-05 10:42:15,004][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][34] overhead, spent [1.7s] collecting in the last [2.2s]
  1&gt; [2016-07-05 10:42:15,004][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][34] overhead, spent [1.7s] collecting in the last [2.2s]
  1&gt; [2016-07-05 10:42:16,731][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][35] overhead, spent [1.3s] collecting in the last [1.7s]
  1&gt; [2016-07-05 10:42:16,731][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][35] overhead, spent [1.3s] collecting in the last [1.7s]
  1&gt; [2016-07-05 10:42:16,731][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][35] overhead, spent [1.3s] collecting in the last [1.7s]
  1&gt; [2016-07-05 10:42:19,183][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][36] overhead, spent [2.2s] collecting in the last [2.4s]
  1&gt; [2016-07-05 10:42:19,183][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][36] overhead, spent [2.2s] collecting in the last [2.4s]
  1&gt; [2016-07-05 10:42:19,184][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][36] overhead, spent [2.2s] collecting in the last [2.4s]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:42:20, stalled for  202s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:42:22,605][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][38] overhead, spent [2.1s] collecting in the last [2.4s]
  1&gt; [2016-07-05 10:42:22,605][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][38] overhead, spent [2.1s] collecting in the last [2.4s]
  1&gt; [2016-07-05 10:42:22,605][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][38] overhead, spent [2.1s] collecting in the last [2.4s]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:42:30, stalled for  212s at: GeoDistanceIT.testDuelOptimizations
  1&gt; [2016-07-05 10:42:32,959][INFO ][org.elasticsearch.monitor.jvm] [node_s2] [gc][old][41][64] duration [7.4s], collections [1]/[8.3s], total [7.4s]/[2.5m], memory [249.4mb]-&gt;[301.3mb]/[491mb], all_pools {[young] [5mb]-&gt;[189.6kb]/[57.5mb]}{[survivor] [56.5mb]-&gt;[0b]/[56.5mb]}{[old] [187.8mb]-&gt;[301.1mb]/[341.5mb]}
  1&gt; [2016-07-05 10:42:32,959][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][41] overhead, spent [7.7s] collecting in the last [8.3s]
  1&gt; [2016-07-05 10:42:32,960][INFO ][org.elasticsearch.monitor.jvm] [node_s1] [gc][old][41][64] duration [7.4s], collections [1]/[8.3s], total [7.4s]/[2.5m], memory [249.4mb]-&gt;[301.3mb]/[491mb], all_pools {[young] [5mb]-&gt;[225kb]/[57.5mb]}{[survivor] [56.5mb]-&gt;[0b]/[56.5mb]}{[old] [187.8mb]-&gt;[301.1mb]/[341.5mb]}
  1&gt; [2016-07-05 10:42:32,960][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][41] overhead, spent [7.7s] collecting in the last [8.3s]
  1&gt; [2016-07-05 10:42:32,960][INFO ][org.elasticsearch.monitor.jvm] [node_s0] [gc][old][41][64] duration [7.4s], collections [1]/[8.3s], total [7.4s]/[2.5m], memory [249.4mb]-&gt;[301.6mb]/[491mb], all_pools {[young] [5mb]-&gt;[464.7kb]/[57.5mb]}{[survivor] [56.5mb]-&gt;[0b]/[56.5mb]}{[old] [187.8mb]-&gt;[301.1mb]/[341.5mb]}
  1&gt; [2016-07-05 10:42:32,960][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][41] overhead, spent [7.7s] collecting in the last [8.3s]
  1&gt; [2016-07-05 10:42:36,837][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][42] overhead, spent [3.1s] collecting in the last [3.8s]
  1&gt; [2016-07-05 10:42:36,837][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][42] overhead, spent [3.1s] collecting in the last [3.8s]
  1&gt; [2016-07-05 10:42:36,837][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][42] overhead, spent [3.1s] collecting in the last [3.8s]
  1&gt; [2016-07-05 10:42:38,746][WARN ][org.elasticsearch.monitor.jvm] [node_s2] [gc][43] overhead, spent [1.5s] collecting in the last [1.9s]
  1&gt; [2016-07-05 10:42:38,746][WARN ][org.elasticsearch.monitor.jvm] [node_s0] [gc][43] overhead, spent [1.5s] collecting in the last [1.9s]
  1&gt; [2016-07-05 10:42:38,746][WARN ][org.elasticsearch.monitor.jvm] [node_s1] [gc][43] overhead, spent [1.5s] collecting in the last [1.9s]
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:42:40, stalled for  222s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:42:50, stalled for  232s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:43:00, stalled for  242s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:43:10, stalled for  252s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:43:20, stalled for  262s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:43:30, stalled for  272s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:43:40, stalled for  282s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:43:50, stalled for  292s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:44:00, stalled for  302s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:44:10, stalled for  312s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:44:21, stalled for  322s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:44:31, stalled for  332s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:44:41, stalled for  342s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:44:51, stalled for  352s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:45:01, stalled for  362s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:45:11, stalled for  372s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:45:21, stalled for  382s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:45:31, stalled for  392s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:45:41, stalled for  402s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:45:51, stalled for  412s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:46:01, stalled for  422s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:46:11, stalled for  432s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:46:21, stalled for  442s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:46:31, stalled for  452s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:46:41, stalled for  462s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:46:51, stalled for  472s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:47:01, stalled for  482s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:47:11, stalled for  492s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:47:21, stalled for  502s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:47:31, stalled for  512s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:47:41, stalled for  522s at: GeoDistanceIT.testDuelOptimizations
HEARTBEAT J0 PID(27139@panthor.local): 2016-07-05T10:47:51, stalled for  532s at: GeoDistanceIT.testDuelOptimizations
</code></pre></div>
<p dir="auto">sometimes it passes but mainly it blows up</p>