title: SOA/Web Service Pagination
body: <p>In SOA we should not be building or holding state (or designing dependencies) between client and server. This is understood. But what patterns can be followed in the case that a client wants to consume a real-time service that may return an open ended number of 'rows'?</p>
  
  <p>Web applications, similar to SOA but allowing for state (sessions) have solved this with pagination. Pagination requires (in most cases, especially with SQL) that the server holds the data and that the client request the data in chunks.</p>
  
  <p>If we where to consider pagination-like scenarios for web services, what patterns would these follow that would still allow the tenets of SOA to be adhered (or as close as possible).</p>
  
  <p>Some rules for the thinkers:
  1) Backed by a SQL database (therefore there is no concept of a row number in a select set)
  2) It is important to not skip a row or duplicate a row in a set during pagination
  3) Data may be inserted and deleted at any time into the database by other clients
  4) There is no need to consider the dataset a live (update-able) dataset</p>
  
  <p>Personally, I think that 1 and 2 above already spell our the solution by constraining the solution space with the requirements. </p>
  
  <p>My proposed solution would have the data (as much as is selected) be stored in a read-only store/cache where it can be assigned a row number within the result set and allow pagination to occur on this data snapshot. I have would have infrastructure to store snapshots (servers, external caches, memcached or ehcache - this must scale quite large). The result of such a query would be a snapshot ID and clients could retrieve the data from the snapshot using a snapshot API (web services) and the snapshot ID. Results would be processed in a read-only, forward only manner for x records at a time where x was something reasonable.</p>
  
  <p>Competing thoughts and ideas, criticisms or accolades would be greatly appreciated. </p>