title: In Logstash, how do I extract fields from a log event using the json filter?
body: <p>Logstash v2.4.1.</p>
  
  <p>I'm sending JSON formatted logs to my Logstash server via UDP packet. The logs look something similar to this.</p>
  
  <pre><code>{
    "key1":"value1",
    "key2":"value2",
    "msg":"2017-03-02 INFO [com.company.app] Hello world"
  }
  </code></pre>
  
  <p>This is my output filter</p>
  
  <pre><code>output {
    stdout {
      codec =&gt; rubydebug
    }
    file {
      path =&gt; "/var/log/trm/debug.log"
      codec =&gt; line { format =&gt; "%{msg}" }
    }
  }
  </code></pre>
  
  <p>The rubydebug output codec shows the log like this</p>
  
  <pre><code>{
    "message" =&gt; {\"key1\":\"value1\", "key2\":\"value2\", \"msg\":\"2017-03-02 INFO [com.company.app] Hello world\"
  }
  </code></pre>
  
  <p>and the file output filter also shows the JSON log correctly, like this</p>
  
  <pre><code>{"key1":"value1", "key2":"value2", "msg":"2017-03-02 INFO [com.company.app] Hello world"}
  </code></pre>
  
  <p>When I use the JSON code in the input filter, I get _jsonparsefailures from Logstash on "some" logs, even though different online JSON parsers parse the JSON correctly, <strong>meaning my logs are in a valid JSON format</strong>.</p>
  
  <pre><code>input {
    udp =&gt; {
      port =&gt; 5555
      codec =&gt; json
    }
  }
  </code></pre>
  
  <p>Therefore, I'm trying to use the json filter instead, like this</p>
  
  <pre><code>filter {
    json =&gt; {
      source =&gt; "message"
    }
  }
  </code></pre>
  
  <p>Using the json filter, how can I extract the "key1", "key2", and the "msg" fields in the "message?"</p>
  
  <p>I tried this to no avail, that is, I don't see the "key1" field in my rubydebug output.</p>
  
  <pre><code>filter {
    json =&gt; {
      source =&gt; "message"
      add_field =&gt; {
        "key1" =&gt; "%{[message][key1]}"
      }
    }
  }
  </code></pre>