Loading…
Glossary term
High-bandwidth memory: stacked memory placed close to AI chips to feed them data quickly.
High-bandwidth memory stacks several memory dies vertically and places them right next to the processor, giving far more data throughput than ordinary memory. AI accelerators carry several HBM stacks.
Because generating text is often limited by how fast weights can be read from memory, HBM matters as much as raw compute.
HBM supply is a key bottleneck in AI hardware, and it competes with consumer devices for memory production.