/tags/database/ Recent content in Database on Taras' Blog on AI, Perf, Hacks /images/papermod-cover.png /images/papermod-cover.png Hugo -- 0.147.0 en Taras Glek Mon, 07 Aug 2023 12:44:29 +0300 /posts/overlooked-hn-databases-that-process-data-faster-than-memory-bandwidth/ Mon, 07 Aug 2023 12:44:29 +0300 /posts/overlooked-hn-databases-that-process-data-faster-than-memory-bandwidth/ <h2 id="13-gibs-per-core">13 GiB/s per core!</h2> <p>Sneller posted a <a href="https://news.ycombinator.com/item?id=36137535">blog on HN</a> on how they use AVX-512 to decompress data at <strong>13 gigabytes</strong> per second per core.</p> <p>This a fantastic ad for their &ldquo;lets turn logs on S3 into cheap database&rdquo; product. This is a solution I wanted multiple times, will definitely consider them next time the need comes up.</p> <h2 id="faster-than-ram">Faster than RAM</h2> <p>Now this post did not get overlooked, but what did get overlooked is that the post engaged the clickhouse CTO. He posted a link to a presentation on how Clickhouse uses compression to process in-memory data faster than <a href="https://presentations.clickhouse.com/meetup53/optimizations/">RAM bandwidth</a>.</p> <p>As a result of discussion in these comments, clickhouse might get even faster.</p>