<p dir="auto">Please allow setting custom analyzers that use the <code class="notranslate">keyword</code> tokenizer and <a href="https://github.com/elastic/elasticsearch/issues/12394#issuecomment-128102606" data-hovercard-type="issue" data-hovercard-url="/elastic/elasticsearch/issues/12394/hovercard">safe token filters</a> on keyword fields.  This would be great so we can use <code class="notranslate">doc_values</code> for things like case-insensitive sorting and icu collation without needing to create pipelines.</p>
<p dir="auto">This was originally discussed in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="96539450" data-permission-text="Title is private" data-url="https://github.com/elastic/elasticsearch/issues/12394" data-hovercard-type="issue" data-hovercard-url="/elastic/elasticsearch/issues/12394/hovercard" href="https://github.com/elastic/elasticsearch/issues/12394">#12394</a> and the last comment from <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/jpountz/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/jpountz">@jpountz</a> said this can be done in a future <code class="notranslate">5.x</code> release.  This issue is to keep track of this potential enhancement.</p>
<p dir="auto">Sometimes you want to analyze text to make it consistent when running aggregations on top of it.</p>
<p dir="auto">For example, let's say I have a <code class="notranslate">city</code> field mapped as a <code class="notranslate">keyword</code>.</p>
<p dir="auto">This field can contain <code class="notranslate">San Francisco</code>, <code class="notranslate">SAN FRANCISCO</code>, <code class="notranslate">San francisco</code>...</p>
<p dir="auto">If I build a terms aggregation on top of it, I will end up with</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="San Francisco: 1
SAN FRANCISCO: 1
San francisco: 1"><pre class="notranslate"><code class="notranslate">San Francisco: 1
SAN FRANCISCO: 1
San francisco: 1
</code></pre></div>
<p dir="auto">I'd like to be able to analyze this text before it gets indexed. Of course I could use a <code class="notranslate">text</code> field instead and set <code class="notranslate">fielddata: true</code> but that would not create doc values for this field.</p>
<p dir="auto">I can imagine that we allow an analyzer at index time for this field.</p>
<p dir="auto">We can restrict its usage if we wish and only allows analyzers which are using tokenizers like <code class="notranslate">lowercase</code>, <code class="notranslate">keyword</code>, <code class="notranslate">path</code> but I would let the user decide.</p>
<p dir="auto">If we allow setting <code class="notranslate">analyzer: simple</code> for example, my aggregation will become:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="san francisco: 3"><pre class="notranslate"><code class="notranslate">san francisco: 3
</code></pre></div>
<p dir="auto">Same applies for path tokenizer.</p>
<p dir="auto">Let say I'm building a dir tree like:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="/tmp/dir1/file1.txt
/tmp/dir1/file2.txt
/tmp/dir2/file3.txt
/tmp/dir2/file4.txt"><pre class="notranslate"><code class="notranslate">/tmp/dir1/file1.txt
/tmp/dir1/file2.txt
/tmp/dir2/file3.txt
/tmp/dir2/file4.txt
</code></pre></div>
<p dir="auto">Applying a path tokenizer would help me to generate an aggregation like:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="/tmp/dir1: 2
/tmp/dir2: 2
/tmp: 4"><pre class="notranslate"><code class="notranslate">/tmp/dir1: 2
/tmp/dir2: 2
/tmp: 4
</code></pre></div>