title: Elasticsearch seemingly random scoring and matching
body: <p>I am using a <code>bool</code> search to match against multiple fields. The fields have been analyzed at index time with multiple filters but mainly use of <code>edge_ngram</code>.</p>
  
  <p>The issue I'm having is that scoring seems to be up in the air. I would expect my search for <code>savvas</code> to match one of my <code>first_name</code> fields of <code>Savvas</code> first, but they are scored much later. For example a search for <code>savvas</code> returns in order of score:</p>
  
  <pre><code>First name | Last name       | Email
  ___________|_________________|________________________
  ------     | Sav---          | ---@sa-------------.com
  -----s     | Sa----          | sa----------s@-----.com
  Sa----     | ----            | sa---------@-------.com  
  Sa----     | --------        | sa-------@---------.com
  sa-        | -----           | sa----------@------.com
  Sa--       | ----s-----s     | sa------s-----s@---.com
  Sa----     | -----------     | sa-----@-----------.com
  Savvas     | -------s        | ----------@--------.com
  Savvas     | -------s        | --------@----------.com
  Sa-        | ---s----S------ | sa------s-----@----.com
  </code></pre>
  
  <p>I have replaced characters other than edge n-grams of the search term in the fields with <code>-</code> and modified the lengths of email to protect identities.</p>
  
  <p>Infact searching for <code>ssssssssssssssss</code> although it doesn't exist in my data returns items with the most number of <code>s</code> characters in them. Something which I wouldn't expect to happen as I am not doing any manual ngrams to my search.</p>
  
  <p>The issue also appears when I attempt to search for a phone number, I match any emails containing the characters <code>78</code> when searching for <code>782</code> over phone numbers that have <code>782</code> as exact ngrams.</p>
  
  <p>It seems that elasticsearch is also performing the ngrams on my search query and not just the field and comparing the two and somehow favouring shorter matches to greater.</p>
  
  <p>Here is my query:</p>
  
  <pre><code>{
      'bool': {
          'should': [ // Any one of these matches will return a result
              {
                  'match': {
                      'phone': {
                          'query': $searchString,
                          'fuzziness': '0',
                          'boost': 3 // If phone matches give it precedence
                      }
                  }
              },
              {
                  'match': {
                      'email': {
                          'query': $searchString,
                          'fuzziness': '0'
                      }
                  }
              },
              {
                  'multi_match': {
                      'query': $searchString,
                      'type': 'cross_fields', // Match if any term is in any of the fields
                      'fields': ['name.first_name', 'name.last_name'],
                      'fuzziness': '0'
                  }
              }
          ],
          'minimum_should_match': 1
      }
  }
  </code></pre>
  
  <p>And the index settings to go with it (apologies for verbosity but I don't want to exclude anything that may be important):</p>
  
  <pre><code>{
      "settings":{
          "analysis":{
              "char_filter":{
                  "trim":{
                      "type":"pattern_replace",
                      "pattern":"^\\s*(.*)\\s*$",
                      "replacement":"$1"
                  },
                  "tel_strip_chars":{
                      "type":"pattern_replace",
                      "pattern":"^(\\(\\d+\\))|^(\\+)|\\D",
                      "replacement":"$1$2"
                  },
                  "tel_uk_exit_coded":{
                      "type":"pattern_replace",
                      "pattern":"^00(\\d+)",
                      "replacement":"+$1"
                  },
                  "tel_parenthesized_country_code":{
                      "type":"pattern_replace",
                      "pattern":"^\\((\\d+)\\)(\\d+)",
                      "replacement":"+$1$2"
                  }
              },
              "tokenizer":{
                  "intl_tel_country_code": {
                      "type":"pattern",
                      "pattern":"\\+(9[976]\\d|8[987530]\\d|6[987]\\d|5[90]\\d|42\\d|3[875]\\d|2[98654321]\\d|9[8543210]|8[6421]|6[6543210]|5[87654321]|4[987654310]|3[9643210]|2[70]|7|1)(\\d{1,14})$",
                      "group":0
                  }
              },
              "filter":{
                  "autocomplete":{
                      "type":"edge_ngram",
                      "min_gram":1,
                      "max_gram":50
                  },
                  "autocomplete_tel":{
                      "type":"ngram",
                      "min_gram":3,
                      "max_gram":20
                  },
                  "email":{
                      "type":"pattern_capture",
                      "preserve_original":1,
                      "patterns":[
                          "([^@]+)",
                          "(\\p{L}+)",
                          "(\\d+)",
                          "@(.+)",
                          "([^-@]+)"
                      ]
                  }
              },
              "analyzer":{
                  "name":{
                      "type":"custom",
                      "tokenizer":"standard",
                      "filter":[
                          "trim",
                          "lowercase",
                          "asciifolding",
                          "autocomplete"
                      ]
                  },
                  "email":{
                      "type":"custom",
                      "tokenizer":"uax_url_email",
                      "filter":[
                          "trim",
                          "lowercase",
                          "email",
                          "unique",
                          "autocomplete"
                      ]
                  },
                  "phone":{
                      "type":"custom",
                      "tokenizer":"intl_tel_country_code",
                      "char_filter":[
                          "trim",
                          "tel_strip_chars",
                          "tel_uk_exit_coded",
                          "tel_parenthesized_country_code"
                      ],
                      "filter":[
                          "autocomplete_tel"
                      ]
                  }
              }
          }
      },
      "mappings":{
          "person":{
              "properties":{
                  "address":{
                      "properties":{
                          "country":{
                              "type":"string",
                              "index_name":"country"
                          }
                      }
                  },
                  "timezone":{
                      "type":"string"
                  },
                  "name":{
                      "properties":{
                          "first_name":{
                              "type":"string",
                              "analyzer":"name"
                          },
                          "last_name":{
                              "type":"string",
                              "analyzer":"name"
                          }
                      }
                  },
                  "email":{
                      "type":"string",
                      "analyzer":"email"
                  },
                  "phone":{
                      "type":"string",
                      "analyzer":"phone"
                  },
                  "id":{
                      "type":"string"
                  }
              }
          }
      }
  }
  </code></pre>
  
  <p>I have tested the index settings using the Kopf plugin's analyser and it appears to create the correct tokens.</p>
  
  <p>Ideally I would only match exactly against the tokens created by my index and prioritise a more precise match in one of my bool's should queries as opposed to prioritising multiple bool shoulds matching.</p>
  
  <p>However I would be happy if atleast it matched exact tokens only. I can't use a <code>term</code> search as my search string itself needs to be tokenized, just without applying any ngrams to it.</p>
  
  <p><strong>To sum up my requirements:</strong></p>
  
  <ul>
  <li>Score first by largest possible match in any single field.</li>
  <li>Then score by lowest offset of possible match in any single field.</li>
  <li>Then score by number of fields matched giving preference to lower offset matches</li>
  </ul>
  
  <p><strong>--- Update: ---</strong></p>
  
  <p>I am getting much better results using a <code>dis_max</code>, it seems to be successfully matching greater ngram matches over multiple ngram matches except for the <code>phone</code> field which is still hard to query. Here's the new query:</p>
  
  <pre><code>{
      'dis_max': {
          'tie_breaker': 0.0,
          'boost': 1.5,
          'queries': [ // Any one of these matches will return a result
              [
                  'match': {
                      'phone': {
                          'query': $searchString,
                          'boost': 1.9
                      }
                  }
              ],
              [
                  'match': {
                      'email': {
                          'query': $searchString
                      }
                  }
              ],
              [
                  'multi_match': {
                      'query': $searchString,
                      'type': 'cross_fields', // Match if any term is in any of the fields
                      'fields': ['name.first_name', 'name.last_name'],
                      'tie_breaker': 0.1,
                      'boost': 1.5
                  }
              ]
          }
      }
  }
  </code></pre>