Cases

One built from zero. Two taken over running. All three measured.

Production systems with revenue depending on them being right. Names of clients and partners left out on purpose; the numbers are the ones we measured.

Built from zero

300B+rows

327.43 billion rows in 2.08 TiB, the largest table 100.78 billion · 2026-09-03

The proof-of-play platform behind 15,000 screens

Designed: One row per ad play, as billing evidence, from screens that lose the network for hours. Playlogs cached on the device, uploaded in batches to an ingestion server, then Kafka into self-managed ClickHouse. MQTT only for presence and telemetry.
As it grew: Two billion rows in MySQL stopped being reasonable, so the store moved to ClickHouse on bare metal. Sort key chosen for the queries first, then one codec pair per column profile, tested over about sixty variations with the heaviest query in the loop.
Running: 200 million rows a day for years. 5.5x overall, 269.6x on the sort-key timestamp, 4.37 bytes per row. The fleet went from the first screen to 15,000 on the same design.

clickhousekafkaedge buffering

Taken over running

460,691

machine-days validated against the source, 18 differences · Oct 2026

Reconciling what was sold with what was on air

As found: Three occupancy engines in the codebase, each with its own idea of a machine-day. Sales wanted a screen; the number behind it couldn't be trusted.
Changed: Merged the engines into one before building anything visible. The screen reads a precomputed snapshot and flags sold-versus-aired divergences. A full month was validated line by line against the source.
Held: 5,392,734 machine-days for 2026, finance matched 100%, and the commercial director compared it to cataract surgery and made it the team's daily routine. Commercial decisions now start from this screen.

reconciliationsnapshotaudit

Taken over running

−67%

telemetry messages per day after the switch · measured on the topic, 2026

Moving device presence to MQTT without losing the fleet

As found: 15,000 screens in elevators and malls announcing themselves over HTTP polling. Losing contact with a screen in the field was the one thing we couldn't afford.
Changed: Four months of testing and a controlled rollout. I set the direction and reviewed every step; a teammate did most of the hands-on work. Proof-of-play kept its own path: cached on the device, uploaded in batches, then Kafka into ClickHouse.
Held: Telemetry volume on Kafka dropped by two thirds with the same fleet, and the billing path stayed exactly as it was.

mqttfleetrollout

The articles go deeper, with the code and the numbers →

Growing faster than your systems?

Let's talk → LinkedIn

Two ways this usually starts. A business with revenue that has to grow without breaking. Or a product that works in the demo and breaks in production. Write me which one. I answer with questions, then with a plan.