<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="&quot;version&quot; : {
    &quot;number&quot; : &quot;1.4.4&quot;,
    &quot;lucene_version&quot; : &quot;4.10.3&quot;
  }"><pre class="notranslate"><code class="notranslate">"version" : {
    "number" : "1.4.4",
    "lucene_version" : "4.10.3"
  }
</code></pre></div>
<p dir="auto">I have about <strong>20m</strong> database of keywords such as:<br>
"<strong>car</strong>" (weight is 100)<br>
"<strong>buy car</strong>" (weight is 50)<br>
"<strong>buy used car</strong>" (weight is 10)<br>
...<br>
each keyword having weight equal popularity. Initial mapping is:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="PUT /suggestions/search/_mapping
{
  &quot;search&quot;: {
            &quot;properties&quot;: {
               &quot;suggest&quot;: {
                  &quot;type&quot;: &quot;completion&quot;,
                  &quot;analyzer&quot;: &quot;standard&quot;
               }
            }
         }
}"><pre class="notranslate"><code class="notranslate">PUT /suggestions/search/_mapping
{
  "search": {
            "properties": {
               "suggest": {
                  "type": "completion",
                  "analyzer": "standard"
               }
            }
         }
}
</code></pre></div>
<p dir="auto">And indexing is simple:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="PUT /suggestions/search/1
{
   &quot;name&quot;: &quot;car&quot;
   &quot;suggest&quot;: {
      &quot;input&quot;: &quot;car&quot;,
      &quot;weight&quot;: 100
   }
}"><pre class="notranslate"><code class="notranslate">PUT /suggestions/search/1
{
   "name": "car"
   "suggest": {
      "input": "car",
      "weight": 100
   }
}
</code></pre></div>
<p dir="auto">So, considering the above list, when I type "car" I will only get "<strong>car</strong>" query. It worked fine as long as I did not want to look up words in middle of phrase. The solution I have found is split keyword to words and add them ti suggestion input as array:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="PUT /suggestions/search/1
{
   &quot;name&quot;: &quot;buy used car&quot;
   &quot;suggest&quot;: {
      &quot;input&quot;: [&quot;buy&quot;, &quot;used&quot;, &quot;car&quot;],
      &quot;weight&quot;: &quot;10&quot;
   }
}"><pre class="notranslate"><code class="notranslate">PUT /suggestions/search/1
{
   "name": "buy used car"
   "suggest": {
      "input": ["buy", "used", "car"],
      "weight": "10"
   }
}
</code></pre></div>
<p dir="auto">Now when I type "car" I can find all of three keywords correctly sorted by weight. The solution was very rapid, much faster then any other elasticsearch autocompletion features. But then there was a big problem.</p>
<p dir="auto">When I reached 200k different keywords related to "car" (indexing arrays like [best, car, buy], [car, rental], [used,car,sale], etc...), I have problem with sorting by weight. When I make query:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="GET /suggestions/_suggest
{
   &quot;text&quot; : &quot;car&quot;,
   &quot;completion&quot;: {
      &quot;field&quot;: &quot;suggest&quot;
   }
}"><pre class="notranslate"><code class="notranslate">GET /suggestions/_suggest
{
   "text" : "car",
   "completion": {
      "field": "suggest"
   }
}
</code></pre></div>
<p dir="auto">The results are sorted in wrong order. The weight (imagine that top keyword "car" has 200k weight and bottom is 1) is shifted randomly and first result has a weight of 100-180k instead of 200k.</p>
<p dir="auto">You can reproduce this by creating index in loop (pseudo-code):</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="for ($i=1;$i&lt;200000;$i++)
{

   PUT /suggestions/search/$i
   {
      &quot;name&quot;: &quot;$i&quot;
      &quot;suggest&quot;: {
         &quot;input&quot;: [&quot;test&quot;, &quot;$i&quot;], // $i as second word is needed otherwise results will be merged
         &quot;weight&quot;: $i
      }
   }

}"><pre class="notranslate"><code class="notranslate">for ($i=1;$i&lt;200000;$i++)
{

   PUT /suggestions/search/$i
   {
      "name": "$i"
      "suggest": {
         "input": ["test", "$i"], // $i as second word is needed otherwise results will be merged
         "weight": $i
      }
   }

}
</code></pre></div>
<p dir="auto">Then try to search "test" and you can see something like this:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
   &quot;_shards&quot;: {
      &quot;total&quot;: 5,
      &quot;successful&quot;: 5,
      &quot;failed&quot;: 0
   },
   &quot;suggest&quot;: [
      {
         &quot;text&quot;: &quot;test&quot;,
         &quot;offset&quot;: 0,
         &quot;length&quot;: 4,
         &quot;options&quot;: [
            {
               &quot;text&quot;: &quot;179992&quot;,
               &quot;score&quot;: 179992
            },
            {
               &quot;text&quot;: &quot;179940&quot;,
               &quot;score&quot;: 179940
            },
            {
               &quot;text&quot;: &quot;179868&quot;,
               &quot;score&quot;: 179868
            },
            {
               &quot;text&quot;: &quot;179820&quot;,
               &quot;score&quot;: 179820
            },
            {
               &quot;text&quot;: &quot;179696&quot;,
               &quot;score&quot;: 179696
            }
         ]
      }
   ]
}"><pre class="notranslate"><code class="notranslate">{
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "suggest": [
      {
         "text": "test",
         "offset": 0,
         "length": 4,
         "options": [
            {
               "text": "179992",
               "score": 179992
            },
            {
               "text": "179940",
               "score": 179940
            },
            {
               "text": "179868",
               "score": 179868
            },
            {
               "text": "179820",
               "score": 179820
            },
            {
               "text": "179696",
               "score": 179696
            }
         ]
      }
   ]
}
</code></pre></div>
<p dir="auto">As you can see, first result is 179992, not 200000. Of course the top weight is 200k and index contains 200k elements. What could be the problem? This is feature of FST working or just a bug?</p>
<p dir="auto"><strong>Elasticsearch version</strong>: Elasticsearch 5.0.0-alpha2</p>
<p dir="auto"><strong>JVM version</strong>: Java 8.0</p>
<p dir="auto"><strong>OS version</strong>: Linux</p>
<p dir="auto"><strong>Description of the problem including expected versus actual behavior</strong>:</p>
<p dir="auto">Based on documentation, I see that we can make use of discovery.ec2.tag in the EC2 discovery plugin as a Prefix to filter down the EC2 instances to be included in the discovery process. I created a tag by name Stage with value as DEV-ELK in EC2 instances in AWS and  I included the following setting in latest Elasticsearch 5.0.0-alpha2 release</p>
<p dir="auto">discovery.ec2.tag.Stage=DEV-ELK</p>
<p dir="auto">But while starting up Elastic Search instance after including the above tag in elasticsearch.yml file, it complains Exception in thread "main" java.lang.IllegalArgumentException: unknown setting [discovery.ec2.tag.Stage]</p>
<p dir="auto"><strong>Steps to reproduce</strong>:<br>
1.Create EC2 instances with Tag as Stage and Value as DEV-ELK<br>
2.Configure elasticsearch.yml file to use Discovery Plugin and include the following setting to filter down the instance discovery.ec2.tag.Stage=DEV-ELK<br>
3.Restart the Elastic Search instance. As soon as you start, you will see the following error pasted in logs section</p>
<p dir="auto"><strong>Provide logs (if relevant)</strong>:</p>
<p dir="auto">Exception in thread "main" java.lang.IllegalArgumentException: unknown setting [discovery.ec2.tag.Stage]<br>
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:267)<br>
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:238)<br>
at org.elasticsearch.common.settings.SettingsModule.configure(SettingsModule.java:120)<br>
at &lt;&lt;&gt;&gt;<br>
at org.elasticsearch.node.Node.(Node.java:235)<br>
at org.elasticsearch.node.Node.(Node.java:161)<br>
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:188)<br>
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:188)<br>
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:267)<br>
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:111)<br>
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:106)<br>
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:91)<br>
at org.elasticsearch.cli.Command.main(Command.java:53)<br>
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:74)<br>
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:67)</p>