Dr Ashish Bamania · GPU Concepts Primer

GPU Concepts for AI Engineers

Nine GPU concepts in four groups: computation hardware covering streaming multiprocessors, CUDA cores across INT32, FP32 and FP64 and tensor cores doing matrix multiply-accumulate; GPU memory with HBM holding weights and activations above an on-chip hierarchy; program execution; and multi-GPU connectivity. The framing is debugging - working out where the compute budget actually goes when a model runs slower than expected. The first four concepts are free to read, five through nine sit behind the paywall.