title: Search engine (elastic search + meteor): Is javascript array manipulation inefficient for arrays containing up to thousands of results?
body: <p>I am working on a project in Meteor which uses ElasticSearch as a search engine. I need the search feature on the site to allow 'stacking' searches. So, for instance, one can search for a file that a user in a certain 'group' uploaded by 'stacking' the user's name, followed by the group name and ending with the file name or some content in the file.</p>
  
  <p>Now, on the MongoDB database the group, user, and files would be stored in separate collections and be related to each other through Ids. However, ElasticSearch uses a distributed datastore where everything is 'flat'. This makes it necessary to denormalize data/do application-side joins/etc. (<a href="https://www.elastic.co/guide/en/elasticsearch/guide/current/relations.html" rel="nofollow">https://www.elastic.co/guide/en/elasticsearch/guide/current/relations.html</a>).</p>
  
  <p><strong>My question is</strong>: which method would be the best...</p>
  
  <ol>
  <li><p>Denormalize data, use nests, etc.</p>
  
  <p>--> So, when rivering data to the elasticsearch datastore, I would make copies of the data and replace every parent element with a new one which has the data added to it.
  FOR EX. If someone comments on let's say a post in a group. The server would have to add to the general list of comments + find the post object, append the comment to it, and re-add the post object to the database + update the group object which contains the post object which should contain the comment + do the same for a user object (since I want to be able to stack searches on groups, users, etc.).
  <strong>Basically</strong> When ever something is added or deleted, I'd have to update every object in the database that relates to it.</p></li>
  <li><p>Run multiple elastic search queries (<a href="https://www.elastic.co/guide/en/elasticsearch/guide/current/application-joins.html" rel="nofollow">https://www.elastic.co/guide/en/elasticsearch/guide/current/application-joins.html</a>) to retrieve the data I want.</p></li>
  <li><p>Just perform search queries on each de-centralized collection, and use javascript on the server-side to compare the arrays and produce the search results.</p></li>
  </ol>
  
  <p>** Note: this is for scaling up to a relatively mid-level load/usage. So around hundreds-thousands of instances of data to search through. Although, if this can work larger scale (millions), that would be great!</p>
  
  <p>Please correct me if my understanding of anything is wrong, and thank you for reading through all this!</p>