title: ElasticSearch Java API get array inside document and apply filter and sort
body: <p>I have to write a complex query, using Java ES API, which will retrieve a page of result, sorted and optionnally filtered. So far here is what my mapping in ElasticSearch looks like :</p>
  
  <pre><code>{
      "mappings" : {
        "applications" : {
          "properties" : {
            "applications" : {
              "properties" : {
                "dynamicSize" : {
                  "type" : "long"
                },
                "name" : {
                  "type" : "string"
                },
                "packageName" : {
                  "type" : "string"
                },
                "type" : {
                  "type" : "string"
                },
                "updateTimestamp" : {
                  "type" : "long"
                },
                "version" : {
                  "type" : "string"
                }
              }
            },
            "imei" : {
              "type" : "string"
            },
            "timestamp" : {
              "type" : "long"
            }
          }
        }
      }
    }
  </code></pre>
  
  <p>Basically, I have a document containing an IMEI (the key to get the document(s)), a timestamp and an applications list. I want to get application lists, for the most recent <strong>timestamp</strong> of the document based on an <strong>imei</strong>. It is hard for me to see how I can write this query. In Java, I've already done this :</p>
  
  <pre><code>SortBuilder sortBuilder = SortBuilders.fieldSort(pageParameter.getOrderField()).order(pageParameter.getOrder());
  SearchHit[] hitsArray = client.prepareSearch(index)
                  .setTypes(type)
                  .setQuery(termQueryBuilder)
                  .setFrom(pageParameter.getFrom())
                  .setSize(pageParameter.getSize())
                  .setPostFilter(pageParameter.getFilters())
                  .addSort(sortBuilder)
                  .execute()
                  .actionGet()
                  .getHits()
                  .getHits();
  </code></pre>
  
  <p>Where pageParameter object contains the <strong>current page</strong> to retrieve (ex : 0 (page 1)), the <strong>size</strong> of the page (ex: 25), the <strong>sortOrder</strong> and a JSONObject of the optionnal <strong>filters</strong>.</p>
  
  <p><strong>The issues here :</strong></p>
  
  <p>1- The problem is that I get my document based on the IMEI, but the sort order won't works as it does not look for a field contained into the "applications" array. </p>
  
  <p>I tried in sort order to put <strong>applications.name</strong> , asc like this :</p>
  
  <pre><code>SortBuilder sortBuilder = SortBuilders.fieldSort("applications." + pageParameter.getOrderField()).order(pageParameter.getOrder());
  </code></pre>
  
  <p>I also added after <strong>setType()</strong> the method <strong>.addField("applications")</strong> because I though it would retrieve me the list, and then I could apply on this my pagination and sorting/filtering process.
  In debug, I see it's all ok the sortBuilder contains "<strong>applications.name</strong>" as field to sort by, and "<strong>asc</strong>" as order</p>
  
  <p>2- And if I get multiple identical IMEI's, I will do this for every document, but I want only the latest.</p>
  
  <p>If someone knows well this API, can you please help me ?
  Thanks !</p>
  
  <p><strong>EDIT :</strong></p>
  
  <p>I tried to do so with at first, a REST request in order to check if this is feasable. At the moment I didn't get the applications list sorted, filtered and paged from the wanted IMEI.</p>
  
  <p>I applied the following mapping : </p>
  
  <pre><code>{
      "mappings": {
          "applications": {
              "properties": {
                  "imei": {
                      "index": "not_analyzed",
                      "type": "string"
                  },
                  "applications": {
                      "type": "nested",
                      "properties": {
                          "dynamicSize": {
                              "type": "long"
                          },
                          "name": {
                              "index": "not_analyzed",
                              "type": "string"
                          },
                          "packageName": {
                              "index": "not_analyzed",
                              "type": "string"
                          },
                          "type": {
                              "index": "not_analyzed",
                              "type": "string"
                          },
                          "version": {
                              "index": "not_analyzed",
                              "type": "string"
                          },
                          "updateTimestamp": {
                              "type": "long"
                          }
                      }
                  },
                  "timestamp": {
                      "type": "long"
                  }
              }
          }
      }
  }
  </code></pre>
  
  <p>After this, I tried different queries, like these ones :</p>
  
  <p><strong>TEST 1 :</strong></p>
  
  <pre><code>{
    "query": {
      "bool": {
        "must": [
          {
            "term": {
              "imei": "359678064430535"
            }
          }
        ]
      }
    },
    "filter": {
        "nested": {
            "path": "applications",
            "filter": {
                "term": {
                    "name" :"SoundAlive"
                }
            }
        }
    },
    "sort": [{
        "applications.name" : { "order" : "asc" }
    }],
    "inner_hits": {},
    "from": 0,
    "size": 5
  }
  </code></pre>
  
  <p><strong>TEST 2 :</strong></p>
  
  <pre><code>{
    "query": {
      "bool": {
        "must": [
          {
            "term": {
              "applications.imei": "359678064430535"
            }
          }
        ],
        "must_not": [],
        "should": []
      }
    },
    "from": 0,
    "size": 10,
    "sort": [
      {
        "applications.name": {
          "order": "asc",
          "nested_path": "applications"
        }
      }
    ],
    "aggs": {}
  }
  </code></pre>
  
  <p><strong>TEST3 :</strong></p>
  
  <pre><code>{
    "query": {
      "bool": {
        "must": [
          {
            "term": {
              "imei": "359678064430535"
            }
          }
        ]
      }
    },
    "sort": [{
        "applications.name" : { "order" : "asc" }
    }],
    "from": 0,
    "size": 2,
    "fields": [ "applications.name" ]
  }
  </code></pre>
  
  <p>In the TEST 3, I'm able to retrieve all the applications.name from the array contained in object, but I can't retrieve the whole object array, nor filter or sort on it :(</p>
  
  <p><strong>EDIT 2 :</strong> </p>
  
  <p>I tried another query today:</p>
  
  <pre><code>{
    "query": {
       "filtered": {
          "query": {
              "match": { 
                "imei" : "359678064430535"
              }
          },
          "filter": {
              "nested": {
                  "path": "applications",
                  "query" : {
                      "match_all": { }
                  },
                  "filter": {
                      "and": [{
                          "term": {
                              "applications.name" : "myAppName"
                          }
                      }]
                  }
              }
          }
        }
    }
  }
  </code></pre>
  
  <p>But I can't sort on a nested query, and the results seems to be insensitive. also I still get the complete object back, not only the application list ... Why is this so hard ? Is it even possible ? </p>