title: Loading a large and dynamically changing data set into a table
body: <p>I have a large set of data that occasionally might be updated in the background. As a self teaching experiment I do not wish to rely on any framework. Right now I am trying to find the right place with the MVC/MVVC patterns within a DDD/CQRS context.</p>
  <p>In order to populate a table I do a query of a large data set and limit the results to twice the number of displayed rows. Once the user scrolls past half of the query, I query for the next batch of results.</p>
  <p>I am loading all results into an array as the user scrolls, just a progressive load without throwing away non displayed rows.</p>
  <p><strong>Who is responsible for building the array of results converted to domain objects? The model of the MVC or the data repository?</strong></p>
  <p>Ideally I would like to reuse my MVC model without being always concerned about limiting the query results.</p>
  <p>If data is updated in the background, a system event is created. However in the cases where a new row was created, I want my application to animate the background insertion of a new row, not just discard the query and reload the table. In such case I guess that a background process must load a copy of the current query and compare it with the existing one. It should create a &quot;New Rows System Event&quot; with the positions of the new rows.</p>
  <p><strong>Which component should compare current query results with previous ones in order to calculate insertion and deletion points? A system service? The MVC model?</strong></p>
  <p>My final concern is that loading a set of rows with multiple limited queries, might be inconsistent if the data is updated in the background. How/Where should I enforce this consistency?</p>