title: Logstash 6.2 - full persistent queue (wrong mapping?)
body: <p>My queue is almost full and I see this errors in my log file:</p>
  
  <pre><code>[2018-05-16T00:01:33,334][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=&gt;400, :action=&gt;["index", {:_id=&gt;nil, :_index=&gt;"2018.05.15-el-mg_papi-prod", :_type=&gt;"doc", :_routing=&gt;nil}, #&lt;LogStash::Event:0x608d85c1&gt;], :response=&gt;{"index"=&gt;{"_index"=&gt;"2018.05.15-el-mg_papi-prod", "_type"=&gt;"doc", "_id"=&gt;"mHvSZWMB8oeeM9BTo0V2", "status"=&gt;400, "error"=&gt;{"type"=&gt;"mapper_parsing_exception", "reason"=&gt;"failed to parse [papi_request_json.query.disableFacets]", "caused_by"=&gt;{"type"=&gt;"i_o_exception", "reason"=&gt;"Current token (VALUE_TRUE) not numeric, can not use numeric value accessors\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@56b8442f; line: 1, column: 555]"}}}}}
  [2018-05-16T00:01:37,145][INFO ][org.logstash.beats.BeatsHandler] [local: 0:0:0:0:0:0:0:1:5000, remote: 0:0:0:0:0:0:0:1:50222] Handling exception: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 69
  [2018-05-16T00:01:37,147][INFO ][org.logstash.beats.BeatsHandler] [local: 0:0:0:0:0:0:0:1:5000, remote: 0:0:0:0:0:0:0:1:50222] Handling exception: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 84
  
  ...
  
  [2018-05-16T15:28:09,981][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=&gt;"cluster_block_exception", "reason"=&gt;"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})
  [2018-05-16T15:28:09,982][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=&gt;"cluster_block_exception", "reason"=&gt;"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})
  [2018-05-16T15:28:09,982][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=&gt;"cluster_block_exception", "reason"=&gt;"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})
  </code></pre>
  
  <p>If I understand first warning, problem is with mapping. I have a lot of files in my queue Logstash folder. My questions is:</p>
  
  <ol>
  <li>How to empty my queue, can i just delete all files from logstash queue folder(all logs will be lost)? And then resend all the data to logstash to proper index?</li>
  <li>How can I determine where exactly is problem in mapping, or which servers sending data of wrong type?</li>
  <li>I have pipeline on port 5000 named testing-pipeline just for checking if Logstash is active from nagios. What is that <code>[INFO ][org.logstash.beats.BeatsHandler]</code> logs?</li>
  <li>If I understand correctly, <code>[INFO ][logstash.outputs.elasticsearch]</code> is just logs about retrying to process logstash queue?</li>
  </ol>
  
  <p>On all servers is FIlebeat 6.2.2. Thank you for your help.</p>