title: Event Consolidation when there is no defined time window for Event Arrival
body: <p>We have a single topic called <strong>migrationstatus</strong>, assume we partition so that all instances and events for a given <strong>MigrationCandidateNumber</strong> always end up on the same topic and partition.</p>
  
  <p>Following event arrives at 12-10-2019 at 10:00 AM</p>
  
  <pre><code>{
  "MigrationCandidateNumber": 54545451,
  "MigrationStatus":"Final Bill Produced"
  }
  </code></pre>
  
  <p>Following event arrives at 14-10-2019 at 08:00 AM</p>
  
  <pre><code>{
  "MigrationCandidateNumber": 54545451,
  "MigrationStatus":"Product Ready"
  }
  </code></pre>
  
  <p>Following event arrives at 17-10-2019 at 12:00 AM </p>
  
  <pre><code>{
  "MigrationCandidateNumber": 54545451,
  "MigrationStatus":"Registration Complete"
  }
  </code></pre>
  
  <p>Problem Statement:</p>
  
  <p>Once all 3 of those events have been processed, we need to produce the event below onto <strong>migrationstatus-out</strong> topic as shown below:</p>
  
  <pre><code>{
      "MigrationCandidateNumber": **54545451**,
      "MigrationStatus":"**Ready for extract 2**"
   }
  </code></pre>
  
  <p>The wide time window is deliberate, since the first 3 events could arrive days apart.</p>
  
  <p>Best way of doing this with no external database?</p>
  
  <p>Solution Tried: </p>
  
  <ol>
  <li>You can't use windowed aggregation because we are not sure about when event arrives. </li>
  <li>created 3 streams out of the main stream for different migration status but again stream-stream joins are windowed. </li>
  <li>For this scenario I don't see a way to aggregate data so that we store data in KSQL table and perform a group by to check If messages with all the status has arrived. </li>
  </ol>
  
  <p>I know its a wide open question and mostly related to approach for solving the problem rather than technical issue but I couldn't find a better forum to post this.  </p>