OXMIQ Simulation Defines HBF's Limit: 14 Times the Capacity at About 60% of Rack Bandwidth
High Bandwidth Flash looks more like a cold-capacity tier beside HBM than a general replacement; MoE experts and sparse long context may fit, but hardware and software are not ready.
Original title: Hot Chips 2026: High Bandwidth Flash promises massive bandwidth and capacity, but its usability is extremely limited — new memory format strikes a balance between HBM and NAND flash
OXMIQ Labs used Hot Chips 2026 to draw a narrower boundary around High Bandwidth Flash than vendor positioning might suggest. In its simulation of a 72-GPU inference rack, an all-HBF configuration offered roughly 14 times the memory capacity of the HBM baseline but only about 60% of its aggregate bandwidth. The result does not show that HBF can generally replace HBM. It suggests a potential role in the smaller set of inference workloads where capacity becomes the constraint before bandwidth does.
HBF packages NAND flash as stacked memory close to an accelerator and connects it through UCIe. The three specification grades reported by Tom's Hardware start at 256GB and 384GB/s and rise to 512GB and 3.072TB/s. Sandisk and SK hynix have released the first technical specification through the Open Compute Project, covering the host interface, electrical behavior, reliability, packaging, and software guidance for reads and writes. Standardization is progressing, but a specification is not evidence that these configurations are already available in purchasable production systems.
OXMIQ's model gives HBF its best chance with infrequently accessed mixture-of-experts weights or parts of a sparse, long-context KV cache, while HBM continues to hold attention weights and hot data. Extra capacity matters when batches are small and a model is otherwise too large for the available memory pool. As batch size and throughput rise, bandwidth demand increases and HBM regains the advantage. ServeTheHome's separate account of the same presentation likewise characterized HBF as a capacity point in the hierarchy, not a cheaper version of HBM.
Software is another gate. The presentation calls for large transfers, direct memory access, and runtime-managed data placement, along with HBF support in inference frameworks such as vLLM. Flash write endurance, asymmetric read-write behavior, and thermal limits also prevent it from acting like transparent DRAM. OXMIQ's results are presenter models and simulations, not third-party measurements on production silicon, and no public HBF supply date is available.
For the storage market, HBF could create a near-compute capacity tier for NAND only if accelerator interfaces, framework scheduling, and suitable workloads mature together. The available evidence does not support an inference that it will displace HBM demand in the near term. The next evidence to watch is working silicon, sustained read and write bandwidth, power and endurance tests, accelerator adoption, framework support, and independent cost-throughput comparisons of the same model on HBM, HBF, and mixed configurations.