title: Finding most similar arrays of integers in elasticsearch
body: <p>REWRITTEN:</p>
  <p>In my project I have images. Each image have 5 tags from range [1,10]. I used Elasticsearch to upload these tags:</p>
  <p>I have these documents loaded into elasticsearch in index &quot;my_project&quot; with type &quot;img&quot;:</p>
  <pre><code>curl -XPUT 'http://localhost:9200/my_project/img/1' -d '
   {&quot;tags&quot;: [1,4,6,7,9]}
  '
  </code></pre>
  <p>Other example documents I upload:</p>
  <pre><code>{&quot;tags&quot;: [1,4,6,7]}
  {&quot;tags&quot;: [2,3,5,6]}
  {&quot;tags&quot;: [1,2,3,8]}
  </code></pre>
  <p>In my application, vectors are much longer, but with fixed number of unique elements. And I have like 20M of these documents.</p>
  <p>Now I want to find similar documents for given vector. Vectors are more similar when they have more common tags. So for example I want to find most similar document for integer vector <code>[1,2,3,7]</code>. The best match should be last example document <code>{&quot;tags&quot;: [1,2,3,8]}</code>, since they share 3 common values in their tags, <code>[1,2,3]</code>, more common values than with any other vectors.</p>
  <p>So here are my problems. If I upload documents with above CURL command, I get this mapping:</p>
  <pre><code>{
    &quot;my_project&quot; : {
      &quot;mappings&quot; : {
        &quot;img&quot; : {
          &quot;properties&quot; : {
            &quot;tags&quot; : {
              &quot;type&quot; : &quot;string&quot;
            }
          }
        }
      }
    }
  }
  </code></pre>
  <p>But I think that correct mapping should use integers instead of strings. How can I make correct explicit mapping for this type of data?</p>
  <p>Now I want to search documents with above similarity algorithm. How can I get 100 most similar documents of above type with similarity algorithm explained above? If I convert these vectors into string with whitespace-separated numbers, I would be able to use boolean query with should statements for this search, but I think that using arrays of integers should be faster. Can you tell me, how can I construct that search query for elasticsearch?</p>
  <hr />
  <h2>My solution so far</h2>
  <p>Basic solution I use now is to convert integer array into string. So I save documents as:</p>
  <pre><code>curl -XPUT 'http://localhost:9200/my_project/img/1' -d '
   {&quot;tags&quot;: &quot;1 4 6 7 9&quot;}
  ' 
  </code></pre>
  <p>and then basically search for string <code>&quot;1 2 3&quot;</code>. While this works somehow, I think that it would be more correct and faster to save array of integers as array of integers, not strings. Is it possible to work with arrays of integers in elasticsearch as with arrays of integers? Maybe my approach with strings is best and can't/don't have to use integer arrays explicitly in elasticsearch.</p>