Benchmarking Lab A field guide to performance Source ↗
Chapter 01Working labMemory access

New to the book? Start here →

Same work,
different order.

We’ll start with a small programming example: two ways to add up the same grid of numbers. You’ll learn how to check the work and measure the time before applying the method to databases.

Your outcome

Run your first benchmark, check that both approaches do the same job, and explain what the measurements support. You can run the example without writing code; the explanations introduce the code you will see. Everything runs on your device.

01 / Inspect

One grid. Two ways through it.

A programming loop repeats a set of instructions. Here, the instructions read a number and add it to a running total. Repeating them lets the program add every number in a grid.

Imagine adding every value in a spreadsheet. You can move across each row, or down each column. Both routes visit every cell once.

We’ll compare loops that follow these two routes. They read the same numbers and return the same sum, so we can focus on one difference: the order of the reads. Does changing that order change how long the work takes? Starting here makes it easier to inspect the work behind the timing before we compare more complex systems.

In memory, our grid is stored as one long array, an ordered sequence of values: the first row, then the second, and so on. This arrangement is called row-major order. Moving across a row reads neighbouring values. Moving down a column jumps by the width of the grid.

The mechanism · illustrative 4 × 4 grid A grid on the page. A sequence in memory.

Array indices in storage order →

Row 00 · 1 · 2 · 3
Row 14 · 5 · 6 · 7
Row 28 · 9 · 10 · 11
Row 312 · 13 · 14 · 15
Across rows

Read the neighbours

0 → 1 → 2 → 3 → …

First four reads: advance one element, or four bytes.

Down columns

Jump by the row width

0 → 4 → 8 → 12 → …

First four reads: advance four elements, or sixteen bytes.

The numbers are addresses expressed as array indices, not stored values or timings. Both complete routes visit all sixteen positions once.

Step through the readsExplanatory model · 4 × 4

Choose an order, then use the arrows or slider to follow each read.

Read order 
for (row = 0; row < n; row++)
  for (col = 0; col < n; col++)
    sum += data[row * n + col];

Illustrative pseudocode. The benchmark runs the WebAssembly source shown below.

Array index · arranged as a grid

Read 1 of 16 · array index 0 · byte offset 0

Current readAlready read

Each value occupies four bytes. This picture shows address order; it does not simulate caches or measure time.

A processor keeps recently used data in small, fast stores called caches. Fetching a value can bring nearby bytes into a cache too. Reading those neighbours soon afterwards can make useful work of that fetch.

This gives us a reason to expect a difference. We still need to measure it. A small grid may fit in cache regardless of traversal order.

02 / Measure

Ask your machine.

The experiment adds every unsigned 32-bit value in a square grid. Both orders use the same Wasm function with different address strides. A stride is the distance between successive elements in a direction.

Keep the work equal

The original scalar Wasm loop reads every cell once per pass. Both orders share one grid; only the address strides change.

Change the question

Choose a grid size and compare the two orders within that run. A larger grid also means more total work.

Experiment 01Measured · WebAssembly

Start with the original source. Keep this tab visible and let the run finish.

Benchmark settings
Inspect or edit the WebAssembly source

WebAssembly text (WAT) describes the function executed by the browser. The four arguments are grid width, repeat count, inner stride, and outer stride. Row order uses strides 1 and n; column order uses n and 1.

Each batch must return the sum of all passes, wrapping at 32 bits. Editing this module changes both traversal orders. A matching checksum checks this fixture; it cannot prove every read took place.

The timing boundary

Each sample times one JavaScript → Wasm call containing a batch of complete grid passes. Setup, warm-up, calibration, checksum checks and drawing sit outside that timer.

Usually a few seconds · stops after 30 seconds

Ready when you are. No timings have been collected.

Before samplingPrepare → warm → calibrate

Fill one shared grid. Warm both orders, then choose a common number of passes per batch.

During samplingTime → check → alternate

Time each batch, verify its sum, and alternate which order runs first in the next pair.

What exactly is being timed?

The worker compiles the module, allocates the grid and fills it with values index % 251 before timing. It warms both orders for at least four pairs, aiming for 100 ms and stopping at 1,000 pairs.

Calibration chooses a common repeat count for both orders. It aims for at least 8 ms in the faster batch, stopping when the slower batch reaches 150 ms or the count reaches 4,096 passes. Calibration timings are discarded.

Each sample times one call from JavaScript into Wasm with performance.now(). Loop overhead and that call are included. Compilation, input setup, correctness checks and drawing are excluded. The starting order is random; subsequent pairs alternate which runs first. Every timed result is checked against the expected unsigned sum.

The timed passes repeatedly read the same grid. This is a warmed, repeated-access experiment. It does not measure cold-cache latency. Browser compilation tiers, scheduling, power management and other programs can still affect timings. Keep this tab visible; leaving it stops the run.

03 / Explain

A result needs its conditions.

Look at the spread of samples as well as the middle value. The median is the middle sample after sorting; with an even count, it is the average of the middle two. It is less affected by one slow interruption than the arithmetic mean.

If row order is faster, that is consistent with benefiting from nearby reads. Timing alone does not identify the cause. Cache behaviour, address translation and browser code generation can all contribute. This lab has no hardware performance counters.

If the samples overlap heavily or a second run changes the ordering, report that uncertainty. The ratio shown here compares two medians from one run; it is not a confidence interval or proof of a general speedup.

The middle
Read each median as milliseconds per complete pass. Lower means less elapsed time for that run’s work.
The spread
Each dot is a batch average per pass. Notice interruptions, clusters and overlap before quoting a ratio.
The ratio
Column median ÷ row median. Above 1 means columns took longer; below 1 means rows took longer. Near 1 means similar medians.
A claim you can defend

“For this grid, code, browser and device, these were the observed times.”

Keep the raw samples and conditions with the claim. The download includes both source and Wasm hashes, settings, batch count, checksums, browser identifier and build.

04 / Experiment again

Try to change your conclusion.

Does the difference depend on how much data you read? Change one condition at a time and keep the evidence from each run.

  1. Make a small-grid record. Run 64 × 64 with the original source and download the result.
  2. Increase the footprint. Run 2,048 × 2,048 with the same source and sample-pair count. Download this result too.
  3. Repeat both runs. Compare the column-to-row ratio and sample spread within each size. Record whether the pattern repeats.

Your task: explain whether traversal order mattered more for one grid on your device. Name one thing these timings cannot establish.

    Compare your explanation

    The larger grid can put more pressure on caches and address translation, so it may show a larger difference. Your device may produce another pattern. Quote your actual medians and spread; a different outcome is not a failed exercise. These runs do not establish native C performance, a cache size, or how another device will behave.

    Break and repair the calculation

    In the Wasm editor, change the first (local.get $sum) inside the addition to (i32.const 0). The loop now replaces its running sum at each read. Run it and inspect the checksum diagnostic. Restore the original source and run again.

    A fast wrong result is not an optimisation. The check catches this deliberate error for this input; it does not certify arbitrary edited code.

    Notes and prediction (optional)

    Source, settings and notes save in this browser. Run records remain in this tab until reload; download any you want to keep.

    Read further

    These references explain the mechanisms and measurement tools used here.