<p dir="auto">If InputRow.getDimensions() has duplicates, IncrementalIndex.add() fails with</p>
<p dir="auto">java.lang.ArrayIndexOutOfBoundsException: 4<br>
at com.metamx.druid.index.v1.IncrementalIndex.add(IncrementalIndex.java:148)<br>
at com.metamx.druid.realtime.Sink.add(Sink.java:98)<br>
at com.metamx.druid.realtime.RealtimeManager$FireChief.run(RealtimeManager.java:176)<br>
[12:29pm]</p>
<p dir="auto">Hi,<br>
We are facing an exception in overlord in a production environment while announcing the Historical Segment.</p>
<p dir="auto">It looks to me some sort of multi-threading issue (if thats possible ) as the segment is already present in the MetaData and still the "segmentExists"  method (IndexerSQLMetadataStorageCoordinator.java+581) is returning "FALSE" on some of the times .<br>
We are using druid-0.9.1 in our production.</p>
<p dir="auto">Following is the Logs :</p>
<p dir="auto">2017-08-08 06:15:02.534+0000 <em>INFO</em>  io.druid.metadata.IndexerSQLMetadataStorageCoordinator Found [prod_report_ds_2017-08-08T05:00:00.000Z_2017-08-08T06:00:00.000Z_2017-08-08T05:00:04.589Z_5] in DB, not updating DB</p>
<p dir="auto">2017-08-08 06:15:01.692+0000 <em>ERROR</em> io.druid.metadata.IndexerSQLMetadataStorageCoordinator Exception inserting segment[prod_report_ds_2017-08-08T05:00:00.000Z_2017-08-08T06:00:00.000Z_2017-08-08T05:00:04.589Z_5] into DB<br>
org.skife.jdbi.v2.exceptions.UnableToExecuteStatementException: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "druid_segments_pkey"<br>
...<br>
Detail: Key (id)=(prod_report_ds_2017-08-08T05:00:00.000Z_2017-08-08T06:00:00.000Z_2017-08-08T05:00:04.589Z_5) already exists.<br>
Show all 89 lines</p>
<p dir="auto">2017-08-08 06:15:01.690+0000 <em>INFO</em> [qtp1997290658-83] io.druid.metadata.IndexerSQLMetadataStorageCoordinator Published segment [prod_report_ds_2017-08-08T05:00:00.000Z_2017-08-08T06:00:00.000Z_2017-08-08T05:00:04.589Z_5] to DB</p>
<p dir="auto">In above logs it is clear the the segment [prod_report_ds_2017-08-08T05:00:00.000Z_2017-08-08T06:00:00.000Z_2017-08-08T05:00:04.589Z_5] has already been inserted in the postgres (metadata) but still overlord throws an exception. Also the very first line (after around 1 minute difference), "segmentExists"  would have returned true.</p>
<p dir="auto">Note the time between last two logger lines , hardly in milliseconds.  Is it possible that two thread would have come at this method (announceHistoricalSegment) and executed segmentExists without knowing about each other (Race Condition)</p>
<p dir="auto">More Details : <a href="https://groups.google.com/forum/#!topic/druid-user/8hFWfsD6THk" rel="nofollow">https://groups.google.com/forum/#!topic/druid-user/8hFWfsD6THk</a></p>