Build your own key-value database
Build a key-value database from scratch. Implement a storage engine with a write-ahead log, indexes and compaction to understand data persistence.
Go · Rust
Build a SQLite reader from scratch. Open a real database file, walk its B-trees and run SELECT queries by decoding the documented binary format.
Build it in your language
The tester treats your program as a black box: any language works. These are just the most convenient ones for this project.
Project in preparation: we are completing and verifying the implementation path and its tests. The capabilities below describe the project goal; starting the project is not enabled yet.
SQLite is probably the most widely deployed piece of software in the world, and its file format is documented down to the last byte. It's the perfect target for learning to read on-disk data structures.
You'll open a real .db — not a toy one — and walk it yourself: header, pages, B-trees, records. You'll finish running SELECT queries with WHERE over real tables, using indexes so you don't scan everything.
10 stages across 4 phases. Every stage comes with its guide and its tests: you move on when they pass.
Header, page size, schema and table listing.
B-tree traversal, varints and record decoding.
A SELECT parser, WHERE filters and column projection.
Index B-trees and queries that avoid the full scan.
Build a key-value database from scratch. Implement a storage engine with a write-ahead log, indexes and compaction to understand data persistence.
Go · Rust
Build a Redis-compatible server from scratch. Implement RESP, an event loop and key expiry, then connect with the official redis-cli.
Go · Rust · Python
Build a vector search engine from scratch. Create a vector index with exact search, filters and a simplified HNSW graph for approximate search.
TypeScript · Python · Go