title: Elasticsearch composite aggregation inner aggregation only returning 10 results
body: <p>I am using elaticsearch <code>v7.1.0</code> to perform composite aggregation to paginate my results.</p>
  <p>The data in the index I am querying and aggregating looks like this.</p>
  <pre><code>{
      &quot;sequence&quot;: &quot;SEQ-A123&quot;,
      &quot;timestamp&quot;: &quot;2022-05-11T12:26:54Z&quot;,
      &quot;owner&quot;: &quot;b96e1abb08d44a6a9871f567aa392167&quot;,
      &quot;serialNo&quot;: &quot;A5645&quot;,
      &quot;value&quot;: 45,
      &quot;ctags&quot;: [
          {
              &quot;name&quot;: &quot;project&quot;,
              &quot;value&quot;: &quot;cd8041f817634e7784b8c0cb5b069d4b&quot;
          }
      ]
  },
  {
      &quot;sequence&quot;: &quot;SEQ-B123&quot;,
      &quot;timestamp&quot;: &quot;2022-05-11T12:26:54Z&quot;,
      &quot;owner&quot;: &quot;b96e1abb08d44a6a9871f567aa392165&quot;,
      &quot;serialNo&quot;: &quot;A8456&quot;,
      &quot;value&quot;: 87,
      &quot;ctags&quot;: [
          {
              &quot;name&quot;: &quot;project&quot;,
              &quot;value&quot;: &quot;cd8041f817634e7784b8c0cb5b069d4b&quot;
          }
      ]
  },
  {
      &quot;sequence&quot;: &quot;SEQ-C123&quot;,
      &quot;timestamp&quot;: &quot;2022-05-11T12:26:54Z&quot;,
      &quot;owner&quot;: &quot;b96e1abb08d44a6a9871f567aa392165&quot;,
      &quot;serialNo&quot;: &quot;A59&quot;,
      &quot;value&quot;: 87,
      &quot;ctags&quot;: [
          {
              &quot;name&quot;: &quot;project&quot;,
              &quot;value&quot;: &quot;cd8041f817634e7784b8c0cb5b069d4b&quot;
          }
      ]
  }, ...
  </code></pre>
  <p>The Query I am executing on elasticsearch is this.</p>
  <pre><code>{
    &quot;query&quot;: {
      &quot;bool&quot;: {
        &quot;must&quot;: [
          {
            &quot;range&quot;: {
              &quot;timestamp&quot;: {
                &quot;gte&quot;: &quot;2022-05-01T00:00:00.000Z&quot;,
                &quot;lte&quot;: &quot;2022-05-30T23:59:59.999Z&quot;
              }
            }
          },
          {
            &quot;terms&quot;: {
              &quot;sequence.keyword&quot;: [
                &quot;SEQ-A123&quot;,
                &quot;SEQ-B123&quot;
              ]
            }
          }
        ],
        &quot;must_not&quot;: [
          {
            &quot;term&quot;: {
              &quot;serialNo.keyword&quot;: &quot;test&quot;
            }
          }
        ]
      }
    },
    &quot;size&quot;: 0,
    &quot;aggs&quot;: {
      &quot;sequence&quot;: {
        &quot;composite&quot;: {
          &quot;sources&quot;: [
            {
              &quot;bkt_sequence&quot;: {
                &quot;terms&quot;: {
                  &quot;field&quot;: &quot;sequence.keyword&quot;
                }
              }
            }
          ],
          &quot;after&quot;: {
            &quot;bkt_sequence&quot;: &quot;&quot;
          },
          &quot;size&quot;: 2
        },
        &quot;aggs&quot;: {
          &quot;serialNo&quot;: {
            &quot;terms&quot;: {
              &quot;field&quot;: &quot;serialNo.keyword&quot;
            },
            &quot;aggs&quot;: {
              &quot;usageStats&quot;: {
                &quot;stats&quot;: {
                  &quot;field&quot;: &quot;value&quot;
                }
              },
              &quot;ctags&quot;: {
                &quot;top_hits&quot;: {
                  &quot;size&quot;: 1,
                  &quot;_source&quot;: {
                    &quot;include&quot;: [
                      &quot;owner&quot;,
                      &quot;ctags&quot;
                    ]
                  }
                }
              }
            }
          }
        }
      }
    }
  }
  </code></pre>
  <p>The result I am getting against this query looks like this.</p>
  <pre><code>{
      &quot;took&quot;: 6,
      &quot;timed_out&quot;: false,
      &quot;_shards&quot;: {
          &quot;total&quot;: 1,
          &quot;successful&quot;: 1,
          &quot;skipped&quot;: 0,
          &quot;failed&quot;: 0
      },
      &quot;hits&quot;: {
          &quot;total&quot;: {
              &quot;value&quot;: 94,
              &quot;relation&quot;: &quot;eq&quot;
          },
          &quot;max_score&quot;: null,
          &quot;hits&quot;: []
      },
      &quot;aggregations&quot;: {
          &quot;sequence&quot;: {
              &quot;after_key&quot;: {
                  &quot;bkt_sequence&quot;: &quot;SEQ-B123&quot;
              },
              &quot;buckets&quot;: [
                  {
                      &quot;key&quot;: {
                          &quot;bkt_sequence&quot;: &quot;SEQ-A123&quot;
                      },
                      &quot;doc_count&quot;: 47,
                      &quot;serialNo&quot;: {
                          &quot;doc_count_error_upper_bound&quot;: 0,
                          &quot;sum_other_doc_count&quot;: 37,
                          &quot;buckets&quot;: [
                              &quot;0&quot;: {
                                  &quot;key&quot;: &quot;A5645&quot;,
                                  &quot;doc_count&quot;: 1,
                                  &quot;ctags&quot;: {
                                      &quot;hits&quot;: {
                                          &quot;total&quot;: {
                                              &quot;value&quot;: 1,
                                              &quot;relation&quot;: &quot;eq&quot;
                                          },
                                          &quot;max_score&quot;: 2,
                                          &quot;hits&quot;: [
                                              &quot;0&quot;: {
                                                  &quot;_index&quot;: &quot;seq-serial&quot;,
                                                  &quot;_type&quot;: &quot;_doc&quot;,
                                                  &quot;_id&quot;: &quot;1&quot;,
                                                  &quot;_score&quot;: 2,
                                                  &quot;_source&quot;: {
                                                      &quot;owner&quot;: &quot;b96e1abb08d44a6a9871f567aa392167&quot;,
                                                      &quot;ctags&quot;: [
                                                          &quot;0&quot;: {
                                                              &quot;name&quot;: &quot;project&quot;,
                                                              &quot;value&quot;: &quot;cd8041f817634e7784b8c0cb5b069d4b&quot;
                                                          }
                                                      ]
                                                  }
                                              }
                                          ]
                                      }
                                  },
                                  &quot;usageStats&quot;: {
                                      &quot;count&quot;: 1,
                                      &quot;min&quot;: 45,
                                      &quot;max&quot;: 45,
                                      &quot;avg&quot;: 45,
                                      &quot;sum&quot;: 45
                                  }
                              },
                              &quot;1&quot;: {
                                  &quot;key&quot;: &quot;A5646&quot;,
                                  &quot;doc_count&quot;: 1,
                                  &quot;ctags&quot;: {
                                      &quot;hits&quot;: {
                                          &quot;total&quot;: {
                                              &quot;value&quot;: 1,
                                              &quot;relation&quot;: &quot;eq&quot;
                                          },
                                          &quot;max_score&quot;: 2,
                                          &quot;hits&quot;: [
                                              &quot;0&quot;: {
                                                  &quot;_index&quot;: &quot;seq-serial&quot;,
                                                  &quot;_type&quot;: &quot;_doc&quot;,
                                                  &quot;_id&quot;: &quot;27&quot;,
                                                  &quot;_score&quot;: 2,
                                                  &quot;_source&quot;: {
                                                      &quot;owner&quot;: &quot;b96e1abb08d44a6a9871f567aa392169&quot;,
                                                      &quot;ctags&quot;: [
                                                          &quot;0&quot;: {
                                                              &quot;name&quot;: &quot;project&quot;,
                                                              &quot;value&quot;: &quot;cd8041f817634e7784b8c0cb5b069d4b&quot;
                                                          }
                                                      ]
                                                  }
                                              }
                                          ]
                                      }
                                  },
                                  &quot;usageStats&quot;: {
                                      &quot;count&quot;: 1,
                                      &quot;min&quot;: 85,
                                      &quot;max&quot;: 85,
                                      &quot;avg&quot;: 85,
                                      &quot;sum&quot;: 85
                                  }
                              },
                              &quot;2&quot;: {
                                  ...
                              },
                              &quot;3&quot;: {
                                  ...
                              },
                              &quot;4&quot;: {
                                  ...
                              },
                              &quot;5&quot;: {
                                  ...
                              },
                              &quot;6&quot;: {
                                  ...
                              },
                              &quot;7&quot;: {
                                  ...
                              },
                              &quot;8&quot;: {
                                  ...
                              },
                              &quot;9&quot;: {
                                  ...
                              }
                          ]
                      }
                  },
                  {
                      &quot;key&quot;: {
                          &quot;bkt_sequence&quot;: &quot;SEQ-B123&quot;
                      },
                      &quot;doc_count&quot;: 47,
                      &quot;serialNo&quot;: {
                          &quot;doc_count_error_upper_bound&quot;: 0,
                          &quot;sum_other_doc_count&quot;: 37,
                          &quot;buckets&quot;: [
                              &quot;0&quot;: {
                                  &quot;key&quot;: &quot;A8456&quot;,
                                  &quot;doc_count&quot;: 1,
                                  &quot;ctags&quot;: {
                                      &quot;hits&quot;: {
                                          &quot;total&quot;: {
                                              &quot;value&quot;: 1,
                                              &quot;relation&quot;: &quot;eq&quot;
                                          },
                                          &quot;max_score&quot;: 2,
                                          &quot;hits&quot;: [
                                              &quot;0&quot;: {
                                                  &quot;_index&quot;: &quot;seq-serial&quot;,
                                                  &quot;_type&quot;: &quot;_doc&quot;,
                                                  &quot;_id&quot;: &quot;48&quot;,
                                                  &quot;_score&quot;: 2,
                                                  &quot;_source&quot;: {
                                                      &quot;owner&quot;: &quot;b96e1abb08d44a6a9871f567aa392167&quot;,
                                                      &quot;ctags&quot;: [
                                                          &quot;0&quot;: {
                                                              &quot;name&quot;: &quot;project&quot;,
                                                              &quot;value&quot;: &quot;cd8041f817634e7784b8c0cb5b069d4b&quot;
                                                          }
                                                      ]
                                                  }
                                              }
                                          ]
                                      }
                                  },
                                  &quot;usageStats&quot;: {
                                      &quot;count&quot;: 1,
                                      &quot;min&quot;: 45,
                                      &quot;max&quot;: 45,
                                      &quot;avg&quot;: 45,
                                      &quot;sum&quot;: 45
                                  }
                              },
                              &quot;1&quot;: {
                                  &quot;key&quot;: &quot;A7590&quot;,
                                  &quot;doc_count&quot;: 1,
                                  &quot;ctags&quot;: {
                                      &quot;hits&quot;: {
                                          &quot;total&quot;: {
                                              &quot;value&quot;: 1,
                                              &quot;relation&quot;: &quot;eq&quot;
                                          },
                                          &quot;max_score&quot;: 2,
                                          &quot;hits&quot;: [
                                              &quot;0&quot;: {
                                                  &quot;_index&quot;: &quot;seq-serial&quot;,
                                                  &quot;_type&quot;: &quot;_doc&quot;,
                                                  &quot;_id&quot;: &quot;74&quot;,
                                                  &quot;_score&quot;: 2,
                                                  &quot;_source&quot;: {
                                                      &quot;owner&quot;: &quot;b96e1abb08d44a6a9871f567aa392169&quot;,
                                                      &quot;ctags&quot;: [
                                                          &quot;0&quot;: {
                                                              &quot;name&quot;: &quot;project&quot;,
                                                              &quot;value&quot;: &quot;cd8041f817634e7784b8c0cb5b069d4b&quot;
                                                          }
                                                      ]
                                                  }
                                              }
                                          ]
                                      }
                                  },
                                  &quot;usageStats&quot;: {
                                      &quot;count&quot;: 1,
                                      &quot;min&quot;: 85,
                                      &quot;max&quot;: 85,
                                      &quot;avg&quot;: 85,
                                      &quot;sum&quot;: 85
                                  }
                              },
                              &quot;2&quot;: {
                                  ...
                              },
                              &quot;3&quot;: {
                                  ...
                              },
                              &quot;4&quot;: {
                                  ...
                              },
                              &quot;5&quot;: {
                                  ...
                              },
                              &quot;6&quot;: {
                                  ...
                              },
                              &quot;7&quot;: {
                                  ...
                              },
                              &quot;8&quot;: {
                                  ...
                              },
                              &quot;9&quot;: {
                                  ...
                              }
                          ]
                      }
                  }
              ]
          }
      }
  }
  </code></pre>
  <p>As you can see there are total 94 documents that are hit with this query.
  47 belongs to the SEQ-A123 bucket and other 47 belongs to SEQ-B123 bucket but out of 47 only 10 documents are returned in the response.</p>
  <p>How can I get all 47 documents in the result and still use pagination at <code>sequence</code> field level?</p>