ch04 · Peak, mean and growth

Builds on ch02 and ch03.

The question

Which number in a demand curve sizes you, and what is a five-year growth rate a claim about?

ch02 established that a workload is a set of quantities. Each of them varies over time, and somebody has collapsed each one into a single number. This chapter is about which collapse is the right one.

The material

The mean is the one number nobody experiences

Demand has a shape. It is low overnight, high in the afternoon, and different again on a Tuesday in November. Sizing for the average means sizing for a level that demand passes through twice a day.

The number that sizes you is the busy hour. Or the busy minute, or the busy Tuesday, depending on how long your system takes to fail and how long anybody will wait for it to recover. Which of those you should use depends on your traffic and your tolerance, and somebody has to decide it.

The arithmetic is trivial: the busiest hour’s share of the day, times the day’s total. Problem 4.1 is that arithmetic. The peak-to-mean ratio behind it is not trivial. It is a measured quantity, it varies by workload, and quoting somebody else’s is how a system gets sized for a shape it does not have.

This chapter puts that ratio into the model as an input with a shape rather than a figure, because nobody has measured it on this service, and the file says so. It then derives the mean rate from the busy hour:

  peak_to_mean:
    kind: input
    decided: world
    unit: dimensionless
    label: peak-to-mean ratio
    distribution:
      triangular:
        minimum: 2.0
        likely: 3.5
        maximum: 6.0
    provenance:
      kind: assumption
      source: >-
        ch04 — the busy hour against the daily mean. Triangular,
        and it is a property of your traffic that belongs to the
        estate target: nothing here can measure it
    range: [1.0, 10.0]
  mean_request_rate:
    kind: derived
    unit: request/second
    label: mean request rate at horizon
    formula: peak_request_rate / peak_to_mean

The busy hour sizes the fleet. The mean is what the fleet spends most of its life serving, and it is the denominator of every cost per request in ch17. That is why the model carries both.

Growth is a bet, and the bet compounds

Growth moves the answer more than any other input in this book.

The chart below is the first of many like it, so here is how it is made. Take one input. Hold every other input still, swing that one from the low end of its range to the high end, and record how far the answer moves. That distance is its swing. Do it for every input and sort the bars longest first. They make a funnel, which is where the name tornado comes from. The two columns in the table are the ends of each swing: low enough that only about one future in ten comes in under, and high enough that only about one in ten comes in over.

Source — web_service-reference · every input on a slider

The growth rate is at the top, and nothing else is close. It is at the top of every tornado in this book whose answer depends on the future. Not every answer does: the cost of a fleet somebody has already bought is a question about prices, and ch15 is where that difference is drawn. But wherever the future is in the chain, growth is at the top of it. That is not a quirk of these numbers. A growth rate is the one input that is raised to a power; everything else is multiplied. Over a five-year horizon, the exponent turns an uncertainty in the rate into a much larger uncertainty in the demand.

That is what a five-year demand forecast looks like when its growth assumption is stated honestly: the busy hour the fleet will have to serve, as a band. The point estimate is somewhere in the middle of it.

The graph has gained two nodes since ch03: the peak-to-mean ratio, and the mean it implies. Three inputs have gained a shape. Click annual growth factor and the band that produced the figure above is written there. Drag its slider and the point moves while the band stays. That is the difference this chapter is about.

Compounding an average is not averaging the compounds

There is a specific and expensive error here. Work through it once by hand.

You have a range of plausible growth rates. You want the capacity in five years. There are two things you could compute:

• take the average growth rate, and compound it; or

• compound every growth rate, and average the results.

They are not the same number. The second is always larger, for any spread at all, because compounding curves upwards. The gap widens with the spread of the growth rates and with the horizon, so it is largest when somebody reaches for a five-year plan.

Problem 4.2 is that comparison. Write down which way you think it goes before you run it.

The practical consequence: a capacity plan built by compounding a single “expected” growth rate understates the expected capacity. Not a pessimistic case: the expected one. The plan is optimistic before any of its other assumptions have been questioned.

What a growth rate is a claim about

Nothing in ch03 helps here.

A compression ratio can be measured. A price can be quoted. A growth rate is a claim about the future, and no amount of provenance discipline turns one into a measurement. The best available version is: the last three years, extrapolated, with a band wide enough to admit that the next three might not resemble them. The width of that band is a judgement nobody can check.

So this book gives growth a lognormal shape: growth compounds, and the multiplier it compounds cannot be zero or less. The ends of the band are stated as a sentence somebody could disagree with: surprised below this, surprised above that. That is the most honest form available, because a sentence can be argued with and a bare number cannot. It is not a measurement, and the model does not pretend otherwise.

Here is that stage of the web service model, with the two ends of a band where a number used to be:

Three ways a demand curve is described badly

A single peak. “We do forty thousand requests a second at peak” is a rate with no duration attached. Forty thousand for ten seconds and forty thousand for four hours size differently. One of them is absorbed by a queue; the other is a queue.

A percentile of the wrong thing. The 95th percentile of per-minute rates across a year is not the busy hour. It is the level one minute in twenty is above. Whether those minutes are scattered evenly through the year or bunched into a few afternoons decides whether sizing to it is right or badly wrong.

A growth rate with no horizon. A growth rate is not an input until somebody says for how long. Over one year it is a rounding error against the other uncertainties. Over five it is the model.

What this cannot tell you

What your peak-to-mean ratio is. Nothing in this repository can measure it. It is a property of your traffic, and it belongs to the estate target (ch03). Everything above tells you what to do with one once you have it.

Whether growth will continue. The model extrapolates. Extrapolation is the assumption that the mechanism producing the last three years is still running. The one thing a capacity model cannot see is the quarter it stops: a product retired, a customer lost, a competitor won.

Whether the band’s width is honest. A growth rate stated as surprised below here, surprised above there is a claim about somebody’s surprise, and nobody goes back afterwards to count how often they were surprised. Nothing here calibrates that.

Anything about a shape that changes. Every figure above assumes demand grows without changing its daily profile. A workload that grows by adding a different kind of user grows in a different shape, and the busy hour moves.

Problems

Three, in tests/peak_mean_and_growth/. The first two have tests. The third does not, and says why.

4.1 — The busy hour. Given a day’s shape as relative weights and a daily total, return the rate during the busiest hour. The weights do not sum to anything in particular, which is most of the problem.

def busy_hour_rate(hourly_shape: np.ndarray, daily_total: float) -> float:
    """Problem 4.1 - the number that sizes you is not the daily mean.

    ``hourly_shape`` is twenty-four relative weights describing how a day's demand is distributed:
    a flat day is twenty-four equal numbers, a spiky one is not. ``daily_total`` is how much
    arrives across the whole day, in whatever unit you like.

    Return the **rate during the busiest hour**, in that unit per hour.

    The weights are relative, not absolute, and they do not necessarily sum to anything in
    particular - normalise them. That is most of the problem, and getting it wrong gives an answer
    that is confidently off by whatever the weights happened to add up to.

    The test also checks a flat day, where the busy-hour rate and the daily mean rate must agree,
    because a system with no peak has nothing to size for.
    """
    raise NotImplementedError("problem 4.1")

4.2 — Compound the average, or average the compounds? Compute both, and find out which is larger and by how much. Predict the direction before you run it. The gap widens with the spread and with the horizon, so it is worst when the plan matters most.

def growth_gap(t0: float, growth_factors: np.ndarray, years: float) -> tuple[float, float]:
    """Problem 4.2 - compound the average, or average the compounds?

    You have a starting capacity ``t0`` and a bag of plausible annual growth factors
    (``growth_factors``: 1.2 means twenty per cent a year). You want the capacity after ``years``.

    There are two things you could compute and people compute the wrong one:

    * take the **average growth rate** and compound it over the years;
    * compound **every** growth rate over the years, and average the results.

    Return both, in that order.

    They are not equal, and one of them is always larger. Work out which before you run it, then
    run it. The gap widens with the spread of the growth rates and with the horizon, which means
    it is largest in exactly the situation people use a five-year plan for.

    The test asserts the order and that the gap is material. It does not tell you the answer; the
    values come from the bag you are handed.
    """
    raise NotImplementedError("problem 4.2")

4.3 — What is your growth rate a claim about? No test. This chapter has already said that nobody can check the width of a growth band, and writing a test for it would contradict that on the same page.

Find the growth rate somebody is currently using to plan the system you work on. Then answer three questions about it. What is it extrapolating: users, requests, retained data, or revenue that somebody has converted into one of those? Over what period was it measured, and is that period long enough to contain the thing that would break it? And what would have to happen for it to be wrong by half, in either direction?

Now write the two ends as a sentence: I would be surprised if it came in under this, and surprised if it came in over that. Say it out loud to whoever owns the plan. The sentence is the deliverable, not the number. A range you are willing to be quoted on is worth more than a point estimate nobody will defend.

A good answer names the mechanism, not just the trend, and its band is wide enough to be uncomfortable. If the band is narrow and comfortable, you have described the last three years rather than the next three.

Where to go next

ch05 begins Part II, and changes the subject from how much demand there is to what happens to a system when it arrives.

ch13 is where the band in this chapter’s second figure comes from. ch19 is what to do about growth sitting at the top of the tornado.