<p dir="auto">The idea behind <code class="notranslate">ignore_malformed</code> setting, is to ensure all records being added get added to the index, even if some of the JSON attribute use an invalid data type (compared to the index's expected mapping). This is particularly useful with dynamic JSON document, where it's most important that all objects are "added" (rather than rejected via exception).</p>
<p dir="auto">However, there are some cases where malformed JSON object are being reject (exception) instead of getting accepted. Repro steps:</p>
<ul dir="auto">
<li>Make sure the <code class="notranslate">ignore_malformed</code> setting is enabled</li>
<li>Add a first record</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="~$ curl -XPUT 'http://127.0.0.1:9200/tweets/tweet/123' -d '{ &quot;value&quot; : 12345 }'
{&quot;_index&quot;:&quot;tweets&quot;,&quot;_type&quot;:&quot;tweet&quot;,&quot;_id&quot;:&quot;123&quot;,&quot;_version&quot;:1,&quot;created&quot;:true}~ $"><pre class="notranslate"><code class="notranslate">~$ curl -XPUT 'http://127.0.0.1:9200/tweets/tweet/123' -d '{ "value" : 12345 }'
{"_index":"tweets","_type":"tweet","_id":"123","_version":1,"created":true}~ $
</code></pre></div>
<p dir="auto">The index now expected a <code class="notranslate">long</code> value for the "value" field.</p>
<ul dir="auto">
<li>Add a malformed record</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="~ $ curl -XPUT 'http://127.0.0.1:9200/tweets/tweet/124' -d '{ &quot;value&quot; : &quot;Hello&quot; }'
{&quot;_index&quot;:&quot;tweets&quot;,&quot;_type&quot;:&quot;tweet&quot;,&quot;_id&quot;:&quot;124&quot;,&quot;_version&quot;:1,&quot;created&quot;:true}"><pre class="notranslate"><code class="notranslate">~ $ curl -XPUT 'http://127.0.0.1:9200/tweets/tweet/124' -d '{ "value" : "Hello" }'
{"_index":"tweets","_type":"tweet","_id":"124","_version":1,"created":true}
</code></pre></div>
<p dir="auto">The malformed record is accepted, as expected.</p>
<ul dir="auto">
<li>Add another malformed record</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="~ $ curl -XPUT 'http://127.0.0.1:9200/tweets/tweet/125' -d '{ &quot;value&quot; : 123123123123123123123 }'
{&quot;error&quot;:&quot;MapperParsingException[failed to parse [value]]; nested: JsonParseException[Numeric value (123123123123123123123) out of range of long (-9223372036854775808 - 9223372036854775807)\n at [Source: [B@52259927; line: 1, column: 34]]; &quot;,&quot;status&quot;:400}"><pre class="notranslate"><code class="notranslate">~ $ curl -XPUT 'http://127.0.0.1:9200/tweets/tweet/125' -d '{ "value" : 123123123123123123123 }'
{"error":"MapperParsingException[failed to parse [value]]; nested: JsonParseException[Numeric value (123123123123123123123) out of range of long (-9223372036854775808 - 9223372036854775807)\n at [Source: [B@52259927; line: 1, column: 34]]; ","status":400}
</code></pre></div>
<p dir="auto">Expected: This malformed record should still be added, with the <code class="notranslate">123123123123123123123</code> value coerced into a <code class="notranslate">string</code>.</p>
<p dir="auto">using elasticsearch 1.5.2 I can index values greater than 127 for a byte field without getting any error. The values just overflow internally and the resulting numbers can be searched for.<br>
e.g. indexing 250 ends up as -6<br>
indexing 260 ends up as 4</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="curl -XPUT localhost:9200/test -d '
{
    &quot;mappings&quot; : {
        &quot;foo&quot; : {
            &quot;properties&quot; : {
                &quot;field1&quot; : { &quot;type&quot; : &quot;byte&quot; }
            }
        }
    }
}'

curl -XPUT localhost:9200/test/foo/one -d '
{
  &quot;field1&quot; : 127
}'

curl -XPUT localhost:9200/test/foo/two -d '
{
  &quot;field1&quot; : 250
}'

curl -XPUT localhost:9200/test/foo/three -d '
{
  &quot;field1&quot; : 260
}'"><pre class="notranslate"><code class="notranslate">curl -XPUT localhost:9200/test -d '
{
    "mappings" : {
        "foo" : {
            "properties" : {
                "field1" : { "type" : "byte" }
            }
        }
    }
}'

curl -XPUT localhost:9200/test/foo/one -d '
{
  "field1" : 127
}'

curl -XPUT localhost:9200/test/foo/two -d '
{
  "field1" : 250
}'

curl -XPUT localhost:9200/test/foo/three -d '
{
  "field1" : 260
}'
</code></pre></div>
<p dir="auto">now searching like this...</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="curl -XGET localhost:9200/test/_search?search_type=count -d '
{
    &quot;aggs&quot; : {
        &quot;values&quot; : {
            &quot;terms&quot; : { &quot;field&quot; : &quot;field1&quot; }
        }
    }
}'"><pre class="notranslate"><code class="notranslate">curl -XGET localhost:9200/test/_search?search_type=count -d '
{
    "aggs" : {
        "values" : {
            "terms" : { "field" : "field1" }
        }
    }
}'
</code></pre></div>
<p dir="auto">returns the following resultset</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
   &quot;took&quot;: 3,
   &quot;timed_out&quot;: false,
   &quot;_shards&quot;: {
      &quot;total&quot;: 5,
      &quot;successful&quot;: 5,
      &quot;failed&quot;: 0
   },
   &quot;hits&quot;: {
      &quot;total&quot;: 3,
      &quot;max_score&quot;: 0,
      &quot;hits&quot;: []
   },
   &quot;aggregations&quot;: {
      &quot;values&quot;: {
         &quot;doc_count_error_upper_bound&quot;: 0,
         &quot;sum_other_doc_count&quot;: 0,
         &quot;buckets&quot;: [
            {
               &quot;key&quot;: -6,
               &quot;doc_count&quot;: 1
            },
            {
               &quot;key&quot;: 4,
               &quot;doc_count&quot;: 1
            },
            {
               &quot;key&quot;: 127,
               &quot;doc_count&quot;: 1
            }
         ]
      }
   }
}"><pre class="notranslate"><code class="notranslate">{
   "took": 3,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hits": {
      "total": 3,
      "max_score": 0,
      "hits": []
   },
   "aggregations": {
      "values": {
         "doc_count_error_upper_bound": 0,
         "sum_other_doc_count": 0,
         "buckets": [
            {
               "key": -6,
               "doc_count": 1
            },
            {
               "key": 4,
               "doc_count": 1
            },
            {
               "key": 127,
               "doc_count": 1
            }
         ]
      }
   }
}
</code></pre></div>
<p dir="auto">trying the same with a 'short' number, you get an error message for document 'two'</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="curl -XPUT localhost:9200/test -d '
{
    &quot;mappings&quot; : {
        &quot;foo&quot; : {
            &quot;properties&quot; : {
                &quot;field1&quot; : { &quot;type&quot; : &quot;short&quot; }
            }
        }
    }
}'


curl -XPUT localhost:9200/test/foo/one -d '
{
  &quot;field1&quot; : 31000
}'

curl -XPUT localhost:9200/test/foo/two -d '
{
  &quot;field1&quot; : 33000
}'"><pre class="notranslate"><code class="notranslate">curl -XPUT localhost:9200/test -d '
{
    "mappings" : {
        "foo" : {
            "properties" : {
                "field1" : { "type" : "short" }
            }
        }
    }
}'


curl -XPUT localhost:9200/test/foo/one -d '
{
  "field1" : 31000
}'

curl -XPUT localhost:9200/test/foo/two -d '
{
  "field1" : 33000
}'
</code></pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
   &quot;error&quot;: &quot;RemoteTransportException[[dev-test-mon-es-02][inet[/192.168.110.83:9300]][indices:data/write/index]]; nested: MapperParsingException[failed to parse [field1]]; nested: JsonParseException[Numeric value (33000) out of range of Java short\n at [Source: UNKNOWN; line: 2, column: 19]]; &quot;,
   &quot;status&quot;: 400
}"><pre class="notranslate"><code class="notranslate">{
   "error": "RemoteTransportException[[dev-test-mon-es-02][inet[/192.168.110.83:9300]][indices:data/write/index]]; nested: MapperParsingException[failed to parse [field1]]; nested: JsonParseException[Numeric value (33000) out of range of Java short\n at [Source: UNKNOWN; line: 2, column: 19]]; ",
   "status": 400
}
</code></pre></div>