title: Fluentd on kubernetes : Segregating container logs based on container name and retag them to send to ElasticSearch
body: <p>I have deployed fluentd in Openshift cluster and setup ES and Kibana On-Premise. I need to capture logs from the nodes and transmit them to ES running on-prem. Specifically, I need to separate the /var/log/containers/*.log into two sections based on a container name (kong)so that all kong logs are tagged as <strong>kong</strong> and remaining ones are tagged as <strong>application</strong>.  Additionally, I would also require the kubernetes metadata info for the pod logs(namespace, container name etc)</p>
  <p>Is there a way to achieve that?</p>
  <p><strong>Fluentd docker image version: fluent/fluentd-kubernetes-daemonset:v1.11.1-debian-elasticsearch7-1.3
  ElasticSearch, Kibana - 7.8.0</strong></p>
  <p>Below are my configuration files.
  <strong>fluentd.conf</strong></p>
  <pre><code>    # AUTOMATICALLY GENERATED
      # DO NOT EDIT THIS FILE DIRECTLY, USE /templates/conf/fluent.conf.erb
  
      @include kubernetes.conf
  
      &lt;match **&gt;
          @type elasticsearch
          @log_level info
          include_tag_key true
          host &quot;#{ENV['FLUENT_ELASTICSEARCH_HOST']}&quot;
          port &quot;#{ENV['FLUENT_ELASTICSEARCH_PORT']}&quot;
          scheme &quot;#{ENV['FLUENT_ELASTICSEARCH_SCHEME'] || 'http'}&quot;
          logstash_format true
          logstash_prefix openshift-${tag}
          user &quot;#{ENV['FLUENT_ELASTICSEARCH_USER'] || use_default}&quot;
          password &quot;#{ENV['FLUENT_ELASTICSEARCH_PASSWORD'] || use_default}&quot;
          &lt;buffer&gt;
           @type file
           path /var/log/fluentd-buffers/kubernetes.system.buffer
           retry_max_interval 30
           flush_interval 1s
           flush_thread_count 8
           chunk_limit_size 2M
           queue_limit_length 32
           overflow_action block
           retry_forever true
          &lt;/buffer&gt;
      &lt;/match&gt; 
  </code></pre>
  <p><strong>kubernetes.conf</strong></p>
  <pre><code>    &lt;label @FLUENT_LOG&gt;
        &lt;match fluent.**&gt;
          @type null
        &lt;/match&gt;
      &lt;/label&gt;
  
      &lt;source&gt;
        @type tail
        path /var/log/containers/*kong*.log
        pos_file /var/log/fluentd-containers.log.pos
        tag &quot;#{ENV['FLUENT_CONTAINER_TAIL_TAG'] || 'kubernetes.*'}&quot;
        exclude_path [&quot;/var/log/containers/fluentd*&quot;]
        read_from_head true
        &lt;parse&gt;
        @type multi_format
          &lt;pattern&gt;
            format json
            time_format '%Y-%m-%dT%H:%M:%S.%N%Z'
            keep_time_key true
          &lt;/pattern&gt;
          &lt;pattern&gt;
            format regexp
            expression /^(?&lt;time&gt;.+) (?&lt;stream&gt;stdout|stderr)( (?&lt;logtag&gt;.))? (?&lt;log&gt;.*)$/
            time_format '%Y-%m-%dT%H:%M:%S.%N%:z'
            keep_time_key true
          &lt;/pattern&gt;
        &lt;/parse&gt;
      &lt;/source&gt;
  
      &lt;filter kubernetes.**&gt;
        @type kubernetes_metadata
        kubernetes_url &quot;#{ENV['FLUENT_FILTER_KUBERNETES_URL'] || 'https://' + ENV.fetch('KUBERNETES_SERVICE_HOST') + ':' + ENV.fetch('KUBERNETES_SERVICE_PORT') + '/api'}&quot;
        verify_ssl &quot;#{ENV['KUBERNETES_VERIFY_SSL'] || true}&quot;
        ca_file &quot;#{ENV['KUBERNETES_CA_FILE']}&quot;
        skip_labels &quot;#{ENV['FLUENT_KUBERNETES_METADATA_SKIP_LABELS'] || 'false'}&quot;
        skip_container_metadata &quot;#{ENV['FLUENT_KUBERNETES_METADATA_SKIP_CONTAINER_METADATA'] || 'false'}&quot;
        skip_master_url &quot;#{ENV['FLUENT_KUBERNETES_METADATA_SKIP_MASTER_URL'] || 'false'}&quot;
        skip_namespace_metadata &quot;#{ENV['FLUENT_KUBERNETES_METADATA_SKIP_NAMESPACE_METADATA'] || 'false'}&quot;
      &lt;/filter&gt;  
  </code></pre>
  <p>I tried changing the tag name <strong>kubernetes.</strong>* to <strong>kong</strong> . I was able to get the log in ES but the kubernetes metadata were missing.</p>
  <p>Please help in this regard.</p>