title: How to parse data from S3 using Logstash and push to Elastic Search and then to Kibana
body: <p>I have a log file created in S3 bucket every minute.
  The data is "\x01" delimited. One of the column is a timestamp field.</p>
  
  <p>I want to load this data into elastic search.</p>
  
  <p>I tried using the following logstash conf. But it doesn't seem to work. I don't see any output. I took some reference from <a href="http://brewhouse.io/blog/2014/11/04/big-data-with-elk-stack.html" rel="nofollow">http://brewhouse.io/blog/2014/11/04/big-data-with-elk-stack.html</a></p>
  
  <p>Logstash config file is as follows:</p>
  
  <pre><code>input {
    s3 {
      bucket =&gt; "mybucketname"
      credentials =&gt; [ "accesskey", "secretkey" ]
    }
  }
  filter {
    csv {
     columns =&gt; [ "col1", "col2", "@timestamp" ]
     separator =&gt; "\x01"
    }
  }
  output {
    stdout { } 
  }
  </code></pre>
  
  <p>How do I modify this file to take in new file coming in every minute? </p>
  
  <p>I would then eventually want to connect Kibana to ES to visualize the changes.</p>