Hi everyone,

I'm learning and exploring topics around database systems. This is a continuation from my last post here: https://www.reddit.com/r/Clickhouse/comments/1x0hu4s/i_tried_to_find_how_clickhouse_keeper_takes/

I think in one of the lectures in CMU's Database Systems course, Prof. Andy had mentioned that ClickHouse has different implementations of hash tables for different datatypes. I wanted to see what these are. So I tried to explore how String Hash Tables are implemented in ClickHouse.

I recorded a concept and code walkthrough here: https://youtu.be/wFCXI9P6Trg

A short summary of the points in the video:

1. Why String keys can hamper performance of hash tables
2. How ClickHouse uses String Adaptive Hash Tables to improve performance
3. The code walk through the files: StringHashTable.h

Do give it a watch and consider supporting by liking the video or subscribing :)

This is partly for my own future reference and partly to share with others who might be interested in it. Would love feedback and corrections from people who know this stuff deeply.

Hi, Im UkJin CHOI from South Korea, Im really passinate for building database engine. Claude code makes it possible. and I found this channel. I worked as DBA and Data engineer for years.

While building many services with PostgreSQL, SQLite, Azure SQL. I found that there are rarely cases in which primary key were update in buisiness logic.

Mutable keys expand undo logs, deepen WAL record and plus complecity.

So Im building a database with immutable tuple key specialized in OLTP database for 2month with Claude code.

Minimum feature were finished like SQLs, Catalog, Transaction (ReadRepeatable, Read Committed), MVCC, WAL, Indexes, Forein Key. Dropping them keeps the engine simpler and faster.

What do you think?

Am I on the proper way? do you think that it will works and company adopt it rather than PostgreSQL if it is mature?

I really appreciate any advice and opinion!

Thank you.

I am the RamCourt author. RamCourt grew out of my years working on RAMCloud at Stateless. RamCourt is an early-stage Rust key-value store: single node so far, with mandatory version-checks and expiry implemented, and replication and recovery not built yet. The part I most want feedback on is the last section: how to order a recreated key against its tombstone when the client-visible version is random, and whether the internal sequence should be per key or table-wide. Happy to be told where I’m wrong. Discussion thread here: https://github.com/OferMania/ramcourt-draft/discussions/7

https://infino.ai/blog/search-index-on-s3/

Read the story of Deepthi Sigireddi and how she came into the world of databases and has spent more than 20 years scaling databases through Vitess at PlanetScale and now Multigres at Supabase. She's among the people who have worked a lot on scaling databases and are still working on some really cool database challenges.

Also, if you love learning about databases, The Consensus is really an amazing resource and has really great blogs about various topics of database systems.

https://theconsensus.dev/p/2026/10/08/deepthi-sigireddi-from-supabase.html?focus=1

I'm using SlateDB myself to save few bucks on RAM, here's some information about how it works internally. It's like a duckdb/leveldb, so it's embeded, but has persistent storage on S3, and I feel like we going into a "Storage Storage everything" era but not all S3 are the same, so there's some bugs I got as well when running under DigitalOcean Spaces

Hi everyone,

I'm learning and exploring topics around database systems. I wanted to see how ClickHouse Keeper takes a non-blocking snapshot of its storage.

I recorded a concept and code walkthrough here: https://www.youtube.com/watch?v=7J93WKEFinA&t

A short summary of the points in the video:

1. Brute force approaches and why they don't work

2. How ClickHouse Keeper uses HashMap + Doubly Linked List to solve this problem

3. The code walk through the files: SnapshotableHashTable.h, KeeperStateMachine.cpp and KeeperSnapshotManager.cpp to see how it's actually implemented.

Do give it a watch and consider supporting by liking the video or subscribing :)

This is partly for my own future reference and partly to share with others who might be interested in it. Would love feedback and corrections from people who know this stuff deeply.

Thank you so much!

MariaDB Foundation continues expanding TAF with PostgreSQL support

More engines, more reproducibility, more value for contributors and the ecosystem.

https://mariadb.org/mariadb-foundation-adds-postgresql-to-its-engine-agnostic-testing-framework-taf/

Have fun benchmarking!

2nd part of the 7-part MECHLOVE Blueprints. All the criticism is extremely welcome (hey, that's the point we're publishing it - to see if it stands).

As a follow-up to previous MECHLOVE Part 1/7: https://6it.dev/blog/radical-mvcc-and-replay-based-rebasing-occ-re2occ-80742

For half a year now, I've been writing my own NoSQL database to store user data instead of relying on regular flat files. Previously, when receiving access credentials for work, saving URLs, or managing other metadata, I had to manually edit text files, which was very inconvenient. I decided to write a utility to handle all my data storage so it could be retrieved quickly whenever needed:

​db create workspace --varchar=256;

db use workspace;

db add login admin;

db add pwd 12345;

db add site example.test;

​db get login; // admin

db open site // open in default browser

db remove pwd;

db drop workspace;

// etc...

​Of course, I won't be using O_DIRECT, AVX, io_uring, or other aggressive performance optimizations here, since it runs locally as a CLI utility rather than a daemon. It's almost a pity, as I've read a ton of material on NoSQL database optimizations, but I plan to apply those techniques in my next project, where I'll focus on a specific storage niche.

​A significant portion of the work is already complete. I plan to release it in early November 2026, accompanied by extensive documentation on the principles and mechanics of NoSQL databases. Out of principle, I wrote everything without using AI/neural networks. This will be my first large-scale project in C.

Hi everyone,

This is a continuation of the explorations from my previous posts here. I went through the CMU Database Systems course, and I'm exploring topics around databases. I wanted to see how the Write Ahead Log is constructed and logged. So I tried to trace a WAL record for a simple INSERT statement in Postgres.

I recorded a video walkthrough of the terminal and source code exploration here: https://youtu.be/YOyq-kvbyU8?si=PTdexP3jmOSdx-8y

I've tried to explore how we can locate a WAL record, how we can decode it via either xxd or pg_waldump, and the relevant source code for it. Do give a watch and consider supporting :)

This is partly for my own future reference and partly to share with others who might be interested in it. Would love feedback and corrections from people who know this stuff deeply. Apologies if this isn't the correct subreddit for this.

Thank you so much!

I am a database researcher and currently conducting some research. I am curious that how to model the hypergraph for no-equi joins involving subqueries, especially for random expressions as join conditions. For example
Select * from A join (select a from B) as C on NULL

We are writing yet another RDBMS, and would appreciate any feedback on our blueprints; while the RDBMS is classical at heart (with WAL and data pages and fuzzy checkpoints), each and every component underneath was redesigned - from WAL being non-ARIES and ZFS-style on-disk CoW to in-memory CoW and cache- and SIMD-friendly page layouts. Our philosophy is playing alongside modern hardware instead of fighting it, MECHLOVE being the highest form of Mechanical Sympathy. As a result, we hope it will beat Hekaton (while being mostly open-source except for certain enterprise features such as HA and at-rest encryption).

Please feel free to comment.

This subreddit is primarily for discussing the implementation of databases, and not about sharing release announcements (either for the first time or your updates).

This thread is the exception!

Please tell us about the new database you (or your agent) built. Tell us about all the cool new features you added. Tell us about anything else you learned or worked on that you haven't gotten around to blogging about yet.

I've just finished on a series covering SQL Server batch mode and columnstore index scan internals.

It's four parts covering:

• Batch Mode
• Columnstore Index Scan internals
• Predicate pushdown
• Aggregate pushdown

AFAIK there's quite a lot in them that hasn't been documented before, especially the row bucketing, which explains the mechanism of the pure vs impure split that is behind a lot of the columnstore optimizations.

How ReductStore uses a persistent store-and-forward queue to deliver edge data over unreliable networks.