GPU Lab CHAPTER 5
How do threads read those values?
Follow five reads from one warp and compare the sectors they touch.
Follow five reads from one warp
PyTorch
import torch
x = torch.arange(1, 26, dtype=torch.float32, device="cuda").reshape(5, 5)
values = x[:, 0]
y = values * 2We keep the same tensor and multiplication. Chapter 4 showed where each value lives. Nearby values can let several threads share a memory request. To see why, first look at how the GPU groups threads that issue a read together.
Definitions
- Warp
- A group of 32 thread positions on an NVIDIA GPU. Threads from the same block are organised into warps; some positions may be unused.
- Lane
- A thread’s position within its warp.
Threads 0–4 read one value each. The other 27 positions make no read in this example. A warp belongs to one block; larger blocks contain several warps.
The GPU handles these reads in fixed-size regions rather than fetching a separate four-byte piece for every thread. Counting the regions touched helps us compare nearby and spread-out inputs.
Definition
- Sector
- An aligned 32-byte region used here to group memory requests. One sector can hold eight FP32 values.
“Aligned” means the regions start at byte offsets 0, 32, 64 and so on. Several threads can read values in the same sector. The diagram below lists the sectors touched and the threads sharing each one.
Run the examples below in your browser with no installation or GPU required. The browser simulates the calculation; it does not execute PyTorch or GPU code. To run the PyTorch snippet itself, use a Python environment with PyTorch and an NVIDIA GPU.
Idea 1 of 1
Compare requests for the same five values
We assign one selected value to each of five threads in the same warp. Each storage allocation starts at a 32-byte boundary in this model. Count the sectors touched by their input read.
Try it
Predict the sector count, then run. Compare row 0, column 0 and its packed copy. Each reads five FP32 values: 20 useful bytes.
PyTorch · operation for these controls
values = x[0, :]
y = values * 2Predict the outputs, then press Run.
Browser simulation · this example keeps its own controls and results.
x: five rows, five columns
Highlighted cells supply the five inputs.
View into x’s storage
- Thread 0
1Element ?
Byte offset ?× 2 →
? - Thread 1
2Element ?
Byte offset ?× 2 →
? - Thread 2
3Element ?
Byte offset ?× 2 →
? - Thread 3
4Element ?
Byte offset ?× 2 →
? - Thread 4
5Element ?
Byte offset ?× 2 →
?
x in memory order
Read left to right, then continue on the next line. Small labels are element offsets from the start of this storage.
Nearby reads can share a sector. These counts describe one input read, excluding the packing copy and output writes. They are not DRAM-read counts or measured time. This thread mapping is a teaching choice; PyTorch may choose a different kernel.
Explain what you found
Try a new case
Predict the sector count for row 1. It still has five neighbouring FP32 values. Must it fit in a single sector? Check with the controls above.
Compare your explanation
Row 1 occupies element offsets 5–9, or bytes 20–39. It crosses the boundary at byte 32, so it touches 2 sectors. Neighbouring values can still cross a sector boundary.
Optional: explore more memory requests
The memory access explorer lets you choose the stride, starting offset and number of active lanes independently. Use it to test patterns beyond our five-value examples. Its sector counts still do not measure GPU time.
You can continue to chapter 6 without doing this extra practice.
Run the example on your desktop
Download this chapter’s opening PyTorch example. See the complete example index and setup appendix for CPU PyTorch, the supported GPGPU-Sim programs and optional NVIDIA GPU execution.
The browser still requires no installation. Downloads are generated from the displayed source; CPU checks and simulator checks do not establish GPU performance.
References
Use these official references to check the concepts and code on this page.
- CUDA Best Practices: coalesced access (new tab)
Compare aligned and misaligned accesses to 32-byte regions.
- CUDA: hardware multithreading (new tab)
Check how a block’s threads are divided into warps.
These browser exercises do not run an installed PyTorch or CUDA toolkit. For optional local execution, choose documentation matching your installed tools. Links open online in a new tab; the questions and answer comparisons below work offline.
Use the documentation
Use the references to explain each answer, then apply it to this page’s example. No installation is needed.