title: How to configure logstash and filebeat SSL communication
body: <h2>The question:</h2>
  
  <p>Can someone help me figure out why I can't get filebeats to talk to logstash over TLS/SSL?</p>
  
  <h2>The Error:</h2>
  
  <p>I can get the filebeat and logstash to talk to eachover with TLS/SSL disabled, but when i enable it and use the settings/config below, I get the following error (observed in <code>logstash.log</code>):</p>
  
  <pre class="lang-ruby prettyprint-override"><code>{:timestamp=&gt;"2016-10-28T17:21:44.445000+0100", :message=&gt;"Pipeline aborted due to error",
   :exception=&gt;java.lang.NullPointerException, :backtrace=&gt;["org.logstash.netty.PrivateKeyCo
  nverter.generatePkcs8(org/logstash/netty/PrivateKeyConverter.java:43)", "org.logstash.nett
  y.PrivateKeyConverter.convert(org/logstash/netty/PrivateKeyConverter.java:39)", "java.lang
  .reflect.Method.invoke(java/lang/reflect/Method.java:498)", "RUBY.create_server(/usr/share
  /logstash/vendor/bundle/jruby/1.9/gems/logstash-input-beats-3.1.0.beta4-java/lib/logstash/
  inputs/beats.rb:139)", "RUBY.register(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/log
  stash-input-beats-3.1.0.beta4-java/lib/logstash/inputs/beats.rb:132)", "RUBY.start_inputs(
  /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:311)", "org.jruby.RubyArray.eac
  h(org/jruby/RubyArray.java:1613)", "RUBY.start_inputs(/usr/share/logstash/logstash-core/li
  b/logstash/pipeline.rb:310)", "RUBY.start_workers(/usr/share/logstash/logstash-core/lib/lo
  gstash/pipeline.rb:187)", "RUBY.run(/usr/share/logstash/logstash-core/lib/logstash/pipelin
  e.rb:145)", "RUBY.start_pipeline(/usr/share/logstash/logstash-core/lib/logstash/agent.rb:2
  40)", "java.lang.Thread.run(java/lang/Thread.java:745)"], :level=&gt;:error}
  {:timestamp=&gt;"2016-10-28T17:21:47.452000+0100", :message=&gt;"stopping pipeline", :id=&gt;"main"
  , :level=&gt;:warn}
  {:timestamp=&gt;"2016-10-28T17:21:47.456000+0100", :message=&gt;"An unexpected error occurred!",
  :error=&gt;#&lt;NoMethodError: undefined method `stop' for nil:NilClass&gt;, :backtrace=&gt;["/us
  r/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-beats-3.1.0.beta4-java/lib/lo
  gstash/inputs/beats.rb:173:in `stop'", "/usr/share/logstash/logstash-core/lib/logstash/inp
  uts/base.rb:88:in `do_stop'", "org/jruby/RubyArray.java:1613:in `each'", "/usr/share/logst
  ash/logstash-core/lib/logstash/pipeline.rb:366:in `shutdown'", "/usr/share/logstash/logsta
  sh-core/lib/logstash/agent.rb:252:in `stop_pipeline'", "/usr/share/logstash/logstash-core/
  lib/logstash/agent.rb:261:in `shutdown_pipelines'", "org/jruby/RubyHash.java:1342:in `each
  '", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:261:in `shutdown_pipelines'",
   "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:123:in `shutdown'", "/usr/share/
  logstash/logstash-core/lib/logstash/runner.rb:237:in `execute'", "/usr/share/logstash/vend
  or/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "/usr/share/logsta
  sh/logstash-core/lib/logstash/runner.rb:157:in `run'", "/usr/share/logstash/vendor/bundle/
  jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'", "/usr/share/logstash/lib/bo
  otstrap/environment.rb:66:in `(root)'"], :level=&gt;:fatal}
  </code></pre>
  
  <h2>The Setup:</h2>
  
  <h3>Servers</h3>
  
  <ul>
  <li><p>2 servers.</p>
  
  <blockquote>
    <p><strong>$> uname -a</strong>
    Linux elkserver 3.10.0-327.36.2.el7.x86_64 #1 SMP Mon Oct 10 23:08:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
    <strong>$> cat /etc/*-release</strong>
    CentOS Linux release 7.2.1511 (Core)</p>
  </blockquote></li>
  <li><p>SELinux is Permissive (soz).</p></li>
  <li>Firewalls are of. (mazza soz).</li>
  <li>One server runs <code>elasticsearch</code> and <code>logstash</code>; one runs <code>filebeat</code>.</li>
  </ul>
  
  <h3>Elasticsearch</h3>
  
  <blockquote>
    <p><strong>$> /usr/share/elasticsearch/bin/elasticsearch -version</strong>
    Version: 2.4.1, Build: c67dc32/2016-09-27T18:57:55Z, JVM: 1.8.0_111</p>
  </blockquote>
  
  <h3>Logstash</h3>
  
  <blockquote>
    <p><strong>$> /usr/share/logstash/bin/logstash -V</strong>
    logstash 5.0.0-alpha5</p>
  </blockquote>
  
  <h3>Filbeat</h3>
  
  <blockquote>
    <p><strong>$> /usr/share/filebeat/bin/filebeat -version</strong>
    filebeat version 5.0.0 (amd64), libbeat 5.0.0</p>
  </blockquote>
  
  <h2>Config:</h2>
  
  <ul>
  <li>Logstash</li>
  </ul>
  
  <blockquote>
  <pre><code>input {
    beats {
      port =&gt; 5044
    ssl =&gt; true
    ssl_certificate =&gt; "/etc/pki/tls/certs/filebeat-forwarder.crt"
    ssl_key =&gt; "/etc/pki/tls/private/filebeat-forwarder.key"
    }
  }
  output {
    elasticsearch {
    hosts =&gt; "localhost:9200"
    manage_template =&gt; false
    index =&gt; "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type =&gt; "%{[@metadata][type]}"
    }
  }
  </code></pre>
  </blockquote>
  
  <ul>
  <li>Filebeat.yml</li>
  </ul>
  
  <blockquote>
  <pre><code>output:
   logstash:
     enabled: true
     hosts:
       - "&lt;my ip address&gt;:5044"
     timeout: 15
     tls:
       certificate_authorities:
       - /etc/pki/tls/certs/filebeat-forwarder.crt
  filebeat:
   prospectors:
     -
       paths:
         - /var/log/syslog
         - /var/log/auth.log
       document_type: syslog
     -
       paths:
         - /var/log/nginx/access.log
       document_type: nginx-access
  </code></pre>
  </blockquote>
  
  <ul>
  <li><p>File: <code>openssl_extras.cnf</code>:</p>
  
  <pre><code>[req]    
  distinguished_name = req_distinguished_name    
  x509_extensions = v3_req    
  prompt = no    
  [req_distinguished_name]    
  C = TG    
  ST = Togo    
  L =  Lome    
  O = Private company    
  CN = *    
  [v3_req]    
  subjectKeyIdentifier = hash    
  authorityKeyIdentifier = keyid,issuer    
  basicConstraints = CA:TRUE    
  subjectAltName = @alt_names        
  [alt_names]    
  DNS.1 = *    
  DNS.2 = *.*    
  DNS.3 = *.*.*    
  DNS.4 = *.*.*.*    
  DNS.5 = *.*.*.*.*    
  DNS.6 = *.*.*.*.*.*    
  DNS.7 = *.*.*.*.*.*.*    
  IP.1 = &lt;my ip address&gt;
  </code></pre></li>
  </ul>
  
  <p>The command used to create the cert:</p>
  
  <blockquote>
    <p>$> openssl req -subj '/CN=elkserver.system.local/' -config /etc/pki/tls/openssl_extras.cnf \
        -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout /etc/pki/tls/private/filebeat-forwarder.key \
        -out /etc/pki/tls/certs/filebeat-forwarder.crt</p>
  </blockquote>