<p dir="auto">I'm an <strong>end user</strong> and I'm new to ES and I've been immediately faced with the problem of getting my data processed correctly.<br>
The problem with ES (vs. Splunk) is that data is being indexed/analyzed first, then stored, then you can't change it. (vs. Splunk, where almost everything happens during search time, thus you can adjust your plans).</p>
<p dir="auto">Currently, in order to test my new mapping params, I have to  (using kopf plugin -- otherwise I'd throw ES immediately out of window for lack of usability):</p>
<ol dir="auto">
<li>Have my data ready -- either stored in some safe index or in local .json file on disk</li>
<li>delete index / index template  -- <code class="notranslate">/_plugin/kopf/#!/cluster</code></li>
<li>adjust index appropriately and create it   -- <code class="notranslate">/_plugin/kopf/#!/createIndex</code></li>
<li>Use <strong>stream2es</strong> in order to copy index from "safe place" to the index above -- <code class="notranslate">cat mydata.json | stream2es  stdin --target http://blah:9200/newidx/type1</code><br>
OR<br>
<code class="notranslate">stream2es es --source http://blah:9200/safe_idx/type1 --target http://blah:9200/newidx/type1</code></li>
<li>Run some query, see it failing/misbehaving, read docs, see that you need mapping adjustment, GO TO step <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="130149" data-permission-text="Title is private" data-url="https://github.com/elastic/elasticsearch/issues/2" data-hovercard-type="issue" data-hovercard-url="/elastic/elasticsearch/issues/2/hovercard" href="https://github.com/elastic/elasticsearch/issues/2">#2</a>.</li>
</ol>
<p dir="auto">Now imagine doing this 50 times, because you can't get ES behave properly with your data....<br>
This is very tedious for someone who expected modern easy-to-use software.</p>
<p dir="auto">Ideally, I want to input my new mapping, press button, and let it do the reindex automatically.<br>
Of course, this would be useful for dev env only as on production, you'd sync the mapping changes with app schema changes.</p>
<p dir="auto">Be able to ask the system to reindex from the saved JSON by document ID or query.  This is useful once we have ES style plugins for manipulating documents that might later change and therefore cause you to want to reindex some set of documents.<br>
<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="405994" data-permission-text="Title is private" data-url="https://github.com/elastic/elasticsearch/issues/490" data-hovercard-type="issue" data-hovercard-url="/elastic/elasticsearch/issues/490/hovercard" href="https://github.com/elastic/elasticsearch/issues/490">#490</a> and <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="405996" data-permission-text="Title is private" data-url="https://github.com/elastic/elasticsearch/issues/491" data-hovercard-type="issue" data-hovercard-url="/elastic/elasticsearch/issues/491/hovercard" href="https://github.com/elastic/elasticsearch/issues/491">#491</a> would let you query by a set of documents indexed before the required change.</p>
<p dir="auto">If you are going to store the JSON, you can take advantage of that by reindex requests.</p>
<p dir="auto">This might also allow the system to handle schema changes in the future more automatically by reindexing to the new analyzer over time in batch.</p>