title: Monitoring log files using some metrics exporter + Prometheus + Grafana
body: <p>I need to monitor very different log files for errors, success status etc. And I need to grab corresponding metrics using Prometheus and show in Grafana + set some alerting on it. Prometheus + Grafana are OK I already use them a lot with different exporters like node_exporter or mysql_exporter etc. Also alerting in new Grafana 4.x works very well.</p>
  
  <p>But I have quite a problem to find suitable exporter/ program which could analyze log files "on fly" and extract metrics from them.</p>
  
  <p>So far I tried:</p>
  
  <ul>
  <li>mtail (<a href="https://github.com/google/mtail" rel="noreferrer">https://github.com/google/mtail</a>) - works but existing version cannot easily monitor more files - in general it cannot bind specific mtail program (receipt for analysis) to some specific log file + I cannot easily add log file name into tag </li>
  <li>grok_exporter (<a href="https://github.com/fstab/grok_exporter" rel="noreferrer">https://github.com/fstab/grok_exporter</a>) - works but I can extract only limited information + one instance can monitor only one log file which mean I would have to start more instances exporting on more ports and configure all off them in prometheus - which makes too many new points of failure</li>
  <li>fluentd prometheus exporter (<a href="https://github.com/kazegusuri/fluent-plugin-prometheus" rel="noreferrer">https://github.com/kazegusuri/fluent-plugin-prometheus</a>) - works but looks like I can extract only very simple metrics and I cannot make any advanced regexp analysis of a line(s) from log file</li>
  </ul>
  
  <p>Does any one here has a really running solution for monitoring advanced metrics from log files using "some exporter" + Prometheus + Grafana? Or instead of exporter some program from which I could grab results using Prometheus push gateway. Thanks.</p>