<story><title>Elastic, Loki and SigNoz – A Perf Benchmark of Open-Source Logging Platforms</title><url>https://github.com/SigNoz/logs-benchmark</url></story><parent_chain></parent_chain><comment><author>trengrj</author><text>I&amp;#x27;m not a fan of competitors creating benchmarks like this as when faced with any tuning decision, they will usually pick the one the makes their competitors slower. But anyway lets take a look at how they tuned Elasticsearch.&lt;p&gt;Disclaimer I &lt;i&gt;used&lt;/i&gt; to work at Elastic!&lt;p&gt;- Used Logstash instead of Beats for simple task of reading syslog json data.
Beats  (&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.elastic.co&amp;#x2F;guide&amp;#x2F;en&amp;#x2F;beats&amp;#x2F;filebeat&amp;#x2F;current&amp;#x2F;filebeat-input-syslog.html&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.elastic.co&amp;#x2F;guide&amp;#x2F;en&amp;#x2F;beats&amp;#x2F;filebeat&amp;#x2F;current&amp;#x2F;fileb...&lt;/a&gt;) would have performed better especially around resource usage.&lt;p&gt;- Set very low Logstash heap of 256mb
&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;SigNoz&amp;#x2F;logs-benchmark&amp;#x2F;blob&amp;#x2F;0b2451e6108d8fa5fdd5808c4e174bd52b9d55d3&amp;#x2F;elk&amp;#x2F;elk-client&amp;#x2F;docker-compose.yaml#LL16C25-L16C29&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;SigNoz&amp;#x2F;logs-benchmark&amp;#x2F;blob&amp;#x2F;0b2451e6108d8f...&lt;/a&gt;&lt;p&gt;- Added grok processor &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;SigNoz&amp;#x2F;logs-benchmark&amp;#x2F;blob&amp;#x2F;0b2451e6108d8fa5fdd5808c4e174bd52b9d55d3&amp;#x2F;elk&amp;#x2F;elk-client&amp;#x2F;pipeline&amp;#x2F;logstash.conf#L9&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;SigNoz&amp;#x2F;logs-benchmark&amp;#x2F;blob&amp;#x2F;0b2451e6108d8f...&lt;/a&gt;
Dissect is faster here&lt;p&gt;- No index template configuration
This would cause higher disk usage than needed due to duplicate mappings. Again a Logstash vs Beats thing. For this test more primary shards and a larger refresh interval would also improve things.&lt;p&gt;- Graph complaining Elasticsearch using 60% available memory.
This is as configured, they could use less with not much impact to performance.&lt;p&gt;- Document counts do not match..
This is probably due to using syslog with random generated data vs creating a test dataset on disk and reading the same data into all platforms.&lt;p&gt;- Aggregation queries were not provided in repo &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;SigNoz&amp;#x2F;logs-benchmark&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;SigNoz&amp;#x2F;logs-benchmark&lt;/a&gt; so cannot validate.&lt;p&gt;I&amp;#x27;m actually surprised Elastic did so well in this benchmark given the misconfiguration.</text></comment>
<story><title>Elastic, Loki and SigNoz – A Perf Benchmark of Open-Source Logging Platforms</title><url>https://github.com/SigNoz/logs-benchmark</url></story><parent_chain></parent_chain><comment><author>wardb</author><text>Unfortunately it&amp;#x27;s severely misunderstood in the benchmark how Grafana Loki should be queried for high cardinality data. See also &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;SigNoz&amp;#x2F;logs-benchmark&amp;#x2F;issues&amp;#x2F;1&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;SigNoz&amp;#x2F;logs-benchmark&amp;#x2F;issues&amp;#x2F;1&lt;/a&gt;</text></comment>