Benchmarking LabA field guide to performanceSource ↗
IntroductionYour guide to the bookAbout 3 minutes

Start here.
Compare with purpose.

A customer asks, “Which technology should we choose?” A performance chart might help—but only if it measures work that matters to them. This book teaches you to design, run and explain comparisons that support that decision.

Your outcome

Build a repeatable benchmark and a concise recommendation supported by evidence. Start with a small experiment you can run in your browser, then develop the skills to design your own comparisons.

01 / The purpose

Give “faster” a useful meaning

Imagine a slide says one database is “twice as fast.” Before using that claim in a customer conversation, you need to know what it did, whether the answers were correct, and how closely the test resembles the customer’s work.

A benchmark is a repeatable test of performance under stated conditions. A useful comparison asks both options to meet the customer’s requirements, records what happened, and gives someone else enough information to check the finding.

You’ll practise turning a broad claim into a specific question: which option completes this work correctly, within the required time, using which resources? Later, you’ll connect that evidence to capacity and cost.

02 / The skills

Run, design and explain

The book is for people who need to compare technology and explain the result, including sales teams working with customers and technical colleagues. You’ll learn to:

  • Run a comparison. Inspect the work, check the answers and collect real measurements.
  • Design your own benchmark. Choose a customer workload, define success and build a repeatable test.
  • Review a claim. Find missing conditions, recognise uncertainty and propose a useful follow-up experiment.
  • Explain a recommendation. Connect performance, cost and tradeoffs to the customer’s decision.

03 / The route

Start small. Build the method.

  1. Try a first experiment. Chapter 1 compares two ways to add a grid of numbers. The work is small enough to follow before you measure it.
  2. Compare real databases. Chapter 2 runs DuckDB and Postgres in your browser. You’ll find an order and total revenue, then examine what the timings support.
  3. Improve the comparison. Chapters 3–8 develop fair tests, representative work, reliable measurements and comparisons under growing demand.
  4. Apply the method more widely. Chapters 9–10 explore Kafka, other data technologies and complete pipelines with several connected parts.
  5. Support a decision. Chapters 11–12 connect the evidence to capacity, total cost of ownership and a recommendation another person can review.

The opening experiments run in your browser. Later chapters combine evidence-analysis tools with downloadable tests against real services. Illustrative exercises are labelled so you can distinguish their inputs from measurements you collect yourself.

04 / Your starting point

Bring curiosity

You don’t need programming or database experience for the opening labs. We’ll introduce the code and technical terms as they become useful. You can run the first examples without writing code, then try changes when you are ready.

Why the labs use WebAssembly

The opening labs use WebAssembly (Wasm), a format that lets compiled programs run inside your browser. It lets us run real code and database engines on your device, without installing software or creating an account.

This makes the experiments easy to try, inspect and repeat. You can follow the work, change its settings and collect your own measurements. Enable JavaScript to run the labs; the chapters remain readable without running an experiment.

These measurements describe your device and browser setup. The infrastructure labs extend the method to real server deployments, using downloadable test programs. A customer’s deployment needs measurements of that deployment.

05 / What you’ll produce

Your finished comparison

As you work through the book, you’ll assemble a short benchmark brief describing the customer’s decision, the work to test and the requirements both options must meet. Alongside it, you’ll keep:

  • A repeatable test with checks that its answers are correct.
  • The raw measurements and the conditions needed to interpret them.
  • A recommendation explaining the findings, costs, tradeoffs and questions still open.

Begin by following two loops through the same numbers. You’ll see why checking the work comes before explaining the time.