Skip to content
Back to the catalog
Project key-value-dbDatabasesAdvancedBeta

A key-value database

A storage engine with a WAL, indexes and compaction. Understand how and when your data is persisted.

Start this challengeBrowse other projects

Build it in your language

GoRust

The tester treats your program as a black box: any language works. These are just the most convenient ones for this project.

Guide language

EnglishEspañol

The stage guides read in this language. Only languages this project is fully translated into show up here; the interface language does not change.

What this project is about

“The database saves it” is one of those phrases we take for granted. This challenge takes it apart: you'll see exactly the moment a piece of data stops living only in memory and survives a power cut.

You'll build an LSM-style storage engine: writes to the log, an in-memory memtable, flushes to sorted files, indexes to read quickly, and compaction so the disk doesn't grow forever. And you'll subject it to brutal crashes to confirm it recovers.

Roadmap

14 stages across 4 phases. Every stage comes with its guide and its tests: you move on when they pass.

  1. Phase 14 stages

    Store and retrieve

    In-memory API, write-ahead log, and recovery by replaying the log at startup.

    1. 01In-memory store
    2. 02Command protocol
    3. 03Write-ahead log
    4. 04WAL recovery
  2. Phase 24 stages

    Down to disk

    Memtable, flushes to immutable sorted files, and merged reads.

    1. 01Sorted memtable
    2. 02Write SSTables
    3. 03Read SSTables
    4. 04Reads across levels
  3. Phase 33 stages

    Reading fast

    Sparse index, Bloom filters and range scans.

    1. 01Sparse index
    2. 02Bloom filter
    3. 03Range scans
  4. Phase 43 stages

    Not growing forever

    Levelled compaction, deletes with tombstones, and measuring the cost.

    1. 01Persistent deletes
    2. 02Levelled compaction
    3. 03Measure the cost

Before you start

An embedded key-value storage engine with a get/put/delete/scan API: a write-ahead log for durability, a sorted memtable, immutable on-disk files with a sparse index, Bloom filters to skip reads, and background compaction.

Related challenges

  • vector-search-engine
    Beta

    Build a vector search engine

    Build a vector index with exact search, filters, and a simplified HNSW graph.

    Advanced5 stages~18 hDatabases
    vectoresbusqueda aproximadahnsw+1

    TypeScript · Python · Go

  • redis
    Beta

    Build your own Redis

    Implement the RESP protocol, an event loop and key expiry. A real server that talks to redis-cli.

    Intermediate10 stages~14 hDatabases
    resptcpeventos+1

    Go · Rust · Python

  • sqlite-reader
    Beta

    Read a SQLite file

    Open a real .db, walk its B-trees and run SELECTs. Reverse engineering of a documented binary format.

    Advanced10 stages~15 hDatabases
    formato binariobtreesql

    Go · Python · Java