title: Elasticsearch in microservices architecture, design question
body: <p>I am designing a system, where I have several microservices communicating via middleware. </p>
  
  <p>Now, every blueprint about microservices underlines that microservices must be autonomous and each of them must handle their own data. Currently, each microservice in my system does store data in a relational database.</p>
  
  <p>I have a new requirement to implement a full-text search, each of my microservices is storing possibly searchable entities. </p>
  
  <p>I was thinking to use an ElasticSearch cluster, where I'd have several indexes, indexes would serve as boundaries that separate the data which comes from various micro-services. I would like to underscore the fact that I plan to use ES only as a search engine, not as a system of record.</p>
  
  <p><em>Here is my dilemma:
  1. Should I allow each microservice to handle ES interactions directly (as caching and persistence)? 2. Or should I create a separate microservice (let's call it "search"), which would be the one that interacts with the ES cluster?
  I am leaning towards 1. b/c since each microservice has to be autonomous in persistence, caching, it can also handle full-text searches too.</em></p>
  
  <p>It will be interesting to hear out different opinions.</p>
  
  <hr>
  
  <p>UPDATE:</p>
  
  <p>Here is why I think each microservice should handle their searches individually:</p>
  
  <ol>
  <li><p>To me, a full-text search capability is similar to persistence and caching layers, each micro-service knows better the business model and is responsible for implementing those layers individually.</p></li>
  <li><p>If I introduce one more microservice just for doing searches, I'll have one extra possible point of failure, same goes to using PubSub as a middleman if we do not want direct interaction between <code>search</code> microservice and the rest of the pack.
  On the contrary, using ES directly, which is a highly-available SaaS, eliminates single point of failure.
  All write requests will be fast and there will be no lag. Information will be searchable right away. This will guarantee a seamless user experience.</p></li>
  <li><p>I do not see search as another business process (maybe my understanding is flawed). To me, it is just a nice-to-have feature, not part of core functionality. However, once implemented, I want it to provide a great user experience.</p></li>
  </ol>
  
  <p>This model of having an individual <code>search</code> microservice reminds of CQRS (command query responsibility segregation) architectural pattern. Where I'd first push the data to DB in my microservice A, then publish it to the messaging broker (command), a message would be picked up from the queue by the consumer and pushed into ES. Then frontend, on the read path (query), would go directly to <code>search</code> microservice.</p>
  
  <p>I have never seen this pattern implemented for searching, it makes sense to do it in a big data world, where one microservice would ingest the data, then the worker process aggregates it for analytics and pushes it into an aggregated data table or separate data store and only then the data will become queryable via separate micro-service, that is enabling fetching of the analytics data.</p>
  
  <p>Are there any publications out there or successful implementations of the CQRS pattern for ES (taking to consideration that ES is not used as a primary system of record but as a full-text search engine)?</p>