03 > Compute Capability

Compute capability is a version number for a GPU generation. It sets the features and hardware limits you met in Lessons 00 to 02, such as the warp size of 32 and the 1024-thread block limit. Every CUDA feature needs a minimum compute capability, so this number tells you what your code may use.

What the Number Means

The format is major.minor, for example 9.0 for Hopper or 8.9 for the L40S used in these lessons. A new major version is a new architecture generation with new hardware. A new minor version is a revision inside the same generation.

Code built for CC 7.0 runs on any GPU with CC 7.0 or higher. Code that uses CC 9.0 features does not run on older GPUs. For example, a program that needs CC 8.0 runs on the L40S (8.9 is higher than 8.0), but a program that needs CC 9.0 does not.

[!TIP] To find the compute capability of your own GPU, run nvidia-smi --query-gpu=name,compute_cap --format=csv.

GPU Generations

The table covers data center GPUs from Pascal to Blackwell, plus the L40S that the outputs in these lessons come from.

SM means Streaming Multiprocessor, the processor inside the GPU that blocks run on. FP32 means 32-bit floating point, and KB means kilobyte.

H100 and B100 have the same per-SM thread and memory limits. The per-SM register count did not change at all across these generations.

[!NOTE] Blackwell is still faster than Hopper because of more SMs (148 on B200 vs 132 on H100 SXM5), 5th generation Tensor Cores, faster HBM3e and NVLink 5.0, NVIDIA's link between GPUs.

Threads per Warp

A warp is a group of 32 threads that the GPU runs together (Lesson 01). The number 32 is fixed by the hardware and is part of the compute capability spec. The GPU never schedules single threads. It always schedules whole warps of 32.

The warp size of 32 has not changed since the first CUDA GPUs (CC 1.0).

Warps and Threads per SM

An SM is the physical processor that blocks run on (Lesson 02). The data center GPUs in the table hold up to 64 active warps per SM, which is 64 x 32 = 2048 threads. The L40S is different. CC 8.9 allows 48 warps per SM, which is 48 x 32 = 1536 threads.

When some warps wait for memory, the warp scheduler can pick other warps. More active warps keep the execution units busy, because there is more often a warp that is ready to run.

[!WARNING] "64 warps per SM" is not true for every GPU. CC 8.6, 8.9 and 12.0 allow only 48. Check the number for your own GPU before you plan around it.

Thread Block Size Limit

In Lesson 02, <<<1, 2048>>> compiled but launched nothing. The reason is the max thread block size of 1024. It is a fixed rule in the compute capability spec, the same 1024 in every column of the table.

The limit is per block, not per SM. One SM can hold more threads than one block may have, as long as they come from several blocks. On the L40S, 1536 threads fit on one SM, for example as 3 blocks of 512 threads. On the A100, 2048 threads fit, for example as 2 blocks of 1024.

FP32 Cores per SM

FP32 is the usual float type. Pascal, Volta, and Ampere data center GPUs have 64 FP32 cores per SM. Ada Lovelace (the L40S), Hopper and Blackwell have 128. More FP32 cores means more floating-point operations per clock cycle on each SM.

Shared Memory per SM

Shared memory is fast memory inside each SM. All threads in a block can use it. It grew over the generations.

• 64 KB on Pascal
• up to 96 KB on Volta
• up to 164 KB on Ampere (A100)
• up to 100 KB on Ada Lovelace (L40S)
• up to 228 KB on Hopper and Blackwell

More shared memory lets a kernel keep more data on-chip instead of going to global memory.

Glossary

• compute capability (CC): a version number (major.minor). It tells which CUDA features a GPU supports and what its hardware limits are.
• SM (Streaming Multiprocessor): the physical processor inside the GPU. All threads run on SMs.
• warp: a group of 32 threads that the GPU schedules and runs together.
• active warps per SM: how many warps one SM can hold at the same time. 64 on most data center GPUs, 48 on CC 8.6, 8.9 and 12.0.
• FP32 (32-bit floating point) core: a hardware unit that does one 32-bit floating-point operation per clock cycle.
• shared memory: fast on-chip memory inside each SM, shared by all threads in a block. Much faster than global (device) memory.
• register: one slot of fast storage in the SM's register file, holding a thread's local variable. An SM has 65536 32-bit registers, and one thread can use at most 255.
• KB (kilobyte): 1024 bytes.
• HBM3e (High Bandwidth Memory 3e): a newer generation of HBM, the fast stacked memory on data center GPUs such as the B200.
• major version: the first number of the CC, such as 8 in 8.9. A new major version means a new architecture generation.
• minor version: the second number of the CC, such as 9 in 8.9. It marks a revision inside the same generation.
• architecture: the design of one GPU generation. NVIDIA names them after scientists, for example Pascal, Volta, Ampere, Ada Lovelace, Hopper, Blackwell.
• nvidia-smi: NVIDIA's command-line tool that comes with the driver. It lists your GPUs, and with --query-gpu=compute_cap it prints their compute capability.
• warp size: the number of threads in a warp, 32 on every NVIDIA GPU so far. A kernel can read it as warpSize.
• warp scheduler: the unit in each SM that picks a ready warp to run. While one warp waits for memory, it switches to another.
• thread block size: the number of threads in one block, the second number in <<<blocks, threads>>>. At most 1024 on every GPU in the table.
• clock cycle: one tick of the processor's clock. At 2 GHz there are 2 billion cycles per second.
• global memory: the large main memory of the GPU, GDDR6 on the L40S and HBM on the H100. Every thread can reach it, but it is much slower than shared memory.
• GPU (Graphics Processing Unit): the processor with thousands of small cores that runs kernels.
• CUDA (Compute Unified Device Architecture): NVIDIA's platform for running your own code on the GPU.