High-bandwidth memory (HBM)
Stacked DRAM chips mounted beside an AI accelerator, giving it much higher memory bandwidth than standard memory modules.
HBM is a type of DRAM built as a vertical stack of memory dies, linked by through-silicon connections and packaged next to a GPU or custom accelerator. The short physical path and very wide interface let the processor move data far faster than conventional memory modules allow. AI training and inference are often limited by memory bandwidth, so HBM is a core input to accelerator performance.
HBM is made by a small number of suppliers, mainly SK hynix, Samsung and Micron, and costs more to produce than standard DRAM. Tight supply or yield problems in HBM can cap how many accelerators a chip vendor ships, even when logic wafers are available. Watch supplier commentary on HBM qualification and its share of memory revenue.
Worked example (illustrative numbers, not real data): an accelerator package uses 6 HBM stacks. If each stack costs 300 dollars to make and sells for 400, the memory gross margin is 25 percent. If the vendor can only get 4 of the 6 stacks it needs, it ships fewer packages from the same wafers.