GPU Lab GPU ARCHITECTURE FOR AI PRACTITIONERS
Follow the data.
Understand the GPU.
You know what a tensor operation does. Find out how threads execute it, where its data moves, and what makes it wait.
You’ll also practise finding and using the official PyTorch and CUDA references to explain an experiment.
Start chapter 1: One operation, many numbers →
Use the GPU map whenever you want to see how concepts fit together. Open the GPU chip, then an SM, block or warp to see how work fits inside the machine. Each concept links to its chapter. The GPU map button opens the map across the reading area, with the contents hidden. Close it to return to the same place and experiment.
Start with five numbers, follow them into GPU memory, then explore how their layout changes the work. You need some Python and tensor familiarity; no CUDA or hardware knowledge is assumed.
THE BOOK AHEAD
From tensor operations to the hardware.
Follow twelve chapters from a tensor operation to GPU execution and AI workloads. Each starts with a small example, then asks you to predict, run and explain. Extra practice is optional. The appendices provide desktop setup and the interactive GPU map.
One operation, many numbers
Follow five values from PyTorch to threads, indices and a small CUDA kernel.
Where do the numbers live?
Copy inputs to the GPU, calculate there, and bring back a result.
How do we select values from a tensor?
Select rows and columns, then multiply the chosen values.
How are tensor values arranged in memory?
Follow values into storage, compare strides, then make a packed copy.
How do threads read those values?
Follow five reads from one warp and compare the sectors they touch.
How can work continue during a wait?
Follow a waiting warp, then see how another warp can make progress.
What if lanes need different calculations?
Follow active lanes through different calculations.
How do threads combine their results?
Build a sum using shared values and block barriers.
How can matrix multiplication reuse its inputs?
Compare repeated loads with shared tiles in matrix multiplication.
What changes when matrix inputs use fewer bits?
Trace input rounding and accumulation, then connect them to matrix hardware.
How can attention store less?
Build a weighted result, then compare full intermediates with tiles.
What limits an inference step?
Follow the KV cache, then compare memory and compute bounds.