What if lanes need different calculations?

Idea 1 of 3

Start with every lane on the same path

All 32 lanes begin with the same sign. A non-empty path issues its operations; an empty path is skipped. In this first experiment each path has one operation, so changing from all positive to all negative changes the selected arithmetic without creating mixed participation.

The dynamic PyTorch example follows the sign control. The diagram highlights the selected operation’s mask; the lane cards include final outputs for checking the calculation.

import torch

values = [2.] * 32
x = torch.tensor(values[:32], device="cuda")
y = torch.where(x > 0, x * 2, -x)
print(y.cpu())

Idea 2 of 3

Let a warp contain both signs

With both signs present, the model runs each non-empty path using its own mask. It issues positive-path operations first, then negative-path operations. Lanes on the other path wait without contributing to the current operation. The displayed order is a teaching choice, not a CUDA guarantee.

Begin with one operation per path. Then change the negative expression from -x to -x * 2 + 1: negate, multiply, add. Even one negative lane causes that non-empty path to appear. Useful lane slots count participating lanes across all issued operations; lane-slot use divides that count by all 32-slot operations.

These counts cover only the listed arithmetic. A compiler can combine operations or execute them conditionally, so this trace does not establish compiled instruction counts or timings.

import torch

values = [2., -2.] * 16
x = torch.tensor(values[:32], device="cuda")
y = torch.where(x > 0, x * 2, -x)
print(y.cpu())

Idea 3 of 3

Separate unused lanes from different paths

Return to the five-value scale of earlier chapters. Here only the first five lanes have inputs. The remaining positions are unused throughout the calculation. That differs from a participating lane masked off for one path that later takes part in its own path.

Keep all inputs positive and vary the input count first. Lane-slot use can fall even when there is only one path. Then keep five inputs and switch to alternating signs to see both effects together. Empty positions in this warp are not filled with another block’s threads; other resident warps may still make progress, as chapter 6 showed.

import torch

values = [2.] * 32
x = torch.tensor(values[:5], device="cuda")
y = torch.where(x > 0, x * 2, -x)
print(y.cpu())