title: ElasticSearch phrase prefix search - How do I get the matched phrase?
body: <p>I'm building an autocomplete feature using ElasticSearch. As the user types, I want to show a list of completions from the data, so the user can select one. For example, if the data contains the following phrases:</p>
  
  <pre><code>very unusual
  very unlikely
  very useful
  </code></pre>
  
  <p>and the user types:</p>
  
  <pre><code>very u
  </code></pre>
  
  <p>I want to display the phrases above.</p>
  
  <p>I'm using this query:</p>
  
  <pre><code>  "query": {
      "multi_match": {
        "query": "very u",
        "fields": [
          "name",
          "description",
          "contentBlocks.caption",
          "contentBlocks.text"
        ],
        "type": "phrase_prefix",
        "max_expansions": 10,
        "cutoff_frequency": 0.001
      }
  </code></pre>
  
  <p>This matches the content I'm looking for, but extracting the matched phrases from the search results is quite awkward. I have been using highlighting, and I collect the matched phrases by parsing the highlights. For example:</p>
  
  <pre><code>    "highlight": {
        "contentBlocks.text": [
          "turned the &lt;em&gt;very&lt;/em&gt; &lt;em&gt;unusual&lt;/em&gt; doorknob"
        ]
      }
  
      "highlight": {
        "contentBlocks.text": [
          "invented a &lt;em&gt;very&lt;/em&gt; &lt;em&gt;useful&lt;/em&gt; mechanism"
        ]
      }
  </code></pre>
  
  <p>What's the right way to do this?</p>
  
  <hr>
  
  <p>"Phrase Suggester" might be capable of doing what I have described, but it is not at all obvious how you would get it to do that.</p>
  
  <p>I have indexed the fields of interest (for example, "description") as follows:</p>
  
  <pre><code>  "description" : {
      "index_analyzer" : "snowball_stem",
      "search_analyzer" : "snowball_stem",
      "type" : "string",
      "fields" : {
        "autocomplete" : {
          "index_analyzer" : "shingle_analyzer",
          "search_analyzer" : "shingle_analyzer",
          "type" : "string"
        }
      }
    },
  </code></pre>
  
  <p>I am using the snowball_stem analyzer for search, and the shingle_analyzer for the autocomplete function. shingle_analyzer looks like this:</p>
  
  <pre><code>"settings" : {
      "analysis" : {
          "analyzer" : {
              "shingle_analyzer" : {
                  "type" : "custom",
                  "tokenizer" : "standard",
                  "filter" : [
                      "standard",
                      "lowercase",
                      "shingle_filter"
                  ],
                  "char_filter" : [
                      "html_strip"
                  ]
              }
          },
          "filter" : {
              "shingle_filter" : {
                  "type" : "shingle",
                  "min_shingle_size" : 2,
                  "max_shingle_size" : 2
              }
          }
      }
  },
  </code></pre>
  
  <p>The documentation for the phrase suggester seems to be totally oriented toward "spelling correction" rather than completion. Since what I'm after is completion, I set the direct generator's min_word_length and prefix_length to the length of the input text, in this case, 2.</p>
  
  <p>I crafted up a suggestion query based on the documentation:</p>
  
  <pre><code>{
      "text" : "sa",
      "autocomplete_description" : {
          "phrase" : {
              "analyzer" : "standard",
              "field" : "description.autocomplete",
              "size" : 10,
              "max_errors" : 2,
              "confidence" : 0.0,
              "gram_size" : 2,
              "direct_generator" : [
                  {
                      "field" : "description.autocomplete",
                      "suggest_mode" : "always",
                      "size" : 10,
                      "min_word_length" : 2,
                      "prefix_length" : 2
                  }
              ]
          }
      }
  }
  </code></pre>
  
  <p>This search for suggestions for "sa" comes up with the following results:</p>
  
  <pre><code>{
    "_shards" : {
      "total" : 1,
      "successful" : 1,
      "failed" : 0
    },
    "autocomplete_description" : [ {
      "text" : "sa",
      "offset" : 0,
      "length" : 2,
      "options" : [ {
        "text" : "say",
        "score" : 0.012580795
      }, {
        "text" : "sa",
        "score" : 0.01127677
      }, {
        "text" : "san",
        "score" : 0.0106529845
      }, {
        "text" : "sad",
        "score" : 0.008533429
      }, {
        "text" : "saw",
        "score" : 0.008107899
      }, {
        "text" : "sam",
        "score" : 0.007155634
      } ]
    } ]
  }
  </code></pre>
  
  <p>What I expect to find for the input "sa" is words that begin with "sa" of any length. Why does it only return words of two or three characters? Why does it only return six options? The multi_match phrase_prefix query I've been using finds many longer words beginning with "sa", such as "saving", "sassy", "safari", and "salad".</p>
  
  <p>When I search for suggestions for multi-word text, such as "one or" (which occurs plenty of times in the data), it finds nothing. The multi_match phrase_prefix query finds "one or more", "one or the", "one, or you", and "one or both".</p>
  
  <p>How can I get this suggester to do what I want?</p>