High Bandwidth Flash (HBF): The New 512GB, 3 TB/s Memory Tier for AI Inference

By AiCybr Editorial · Published August 7, 2026

Sandisk and SK hynix have moved High Bandwidth Flash (HBF) from an idea into a defined memory standard aimed at AI inference systems.

At FMS 2026, the companies formally introduced the first HBF specification. The initial targets are striking: packages of up to 512GB and bandwidth grades ranging from roughly 0.4 TB/s to 3.0 TB/s.

HBF is not intended to replace HBM outright, and it is not simply a very fast SSD. It is a new NAND-based memory tier designed to sit much closer to AI accelerators, trading the ultra-low latency of DRAM-based HBM for dramatically higher capacity, persistence, and potentially lower cost per gigabyte.

That matters because modern AI inference is increasingly constrained not only by compute, but by how much model state can be kept close enough to the accelerator to avoid expensive data movement.

Validation note: This guide was checked on August 7, 2026. Sandisk and SK hynix have publicly committed to HBF standardization under the Open Compute Project, and the first specification targets have now been disclosed. However, HBF is still an emerging technology: the specification is not yet broadly published through OCP, commercial shipping products are not yet generally available, and real-world latency, sustained bandwidth, endurance, thermals, pricing, and software behavior remain to be independently benchmarked.

Navigate This Guide

• HBF at a glance
• What High Bandwidth Flash is
• Why AI inference needs another memory tier
• HBF vs HBM vs DRAM vs SSD
• The first HBF specification
• How HBF is expected to connect to accelerators
• Why 512GB per package matters
• What 3 TB/s really means
• Where HBF could help AI inference
• HBF and KV-cache-heavy workloads
• Could HBF help local AI
• What HBF does not solve
• Adoption and ecosystem risks
• Roadmap and availability
• Frequently asked questions
• Final assessment
• Sources

HBF at a Glance

The central idea is simple:

HBM is extremely fast but expensive and capacity-limited. SSDs are extremely capacious but too far from the accelerator for many latency-sensitive inference paths. HBF tries to occupy the space between them.

What High Bandwidth Flash Is

High Bandwidth Flash is a new memory category built from NAND flash and designed for direct, high-bandwidth attachment to AI accelerators and other high-performance compute devices.

Sandisk and SK hynix began formal standardization work in February 2026 under an Open Compute Project workstream. Their stated goal was to create a memory layer between HBM and SSD storage that could provide:

• much higher capacity than HBM;
• much higher bandwidth than conventional SSD storage;
• non-volatility;
• lower cost per bit than DRAM-based memory;
• scalability for AI inference systems.

That design goal matters because AI systems increasingly contain several distinct memory tiers rather than a single pool.

A simplified hierarchy may look like this:

On-chip SRAM / cache → HBM → HBF → SSD → remote storage

Each step trades some combination of latency and bandwidth for capacity and cost.

HBF is trying to create a useful new point on that curve.

Why AI Inference Needs Another Memory Tier

Training and inference stress memory differently.

Large-scale training requires immense bandwidth for repeatedly moving parameters, activations and optimizer state while many accelerators work together.

Inference often has a different problem: the model or serving state must remain available continuously while user requests arrive, and the amount of relevant data can become enormous.

Three trends are especially important.

1. Models are larger

Even when quantized, frontier-class models can occupy hundreds of gigabytes or more.

An illustrative memory requirement for model weights alone is:

These figures exclude runtime overhead, activations, KV cache, temporary buffers and framework allocations.

A single 512GB HBF package therefore enters a capacity range that is meaningful for very large quantized models or large pools of reusable inference state.

2. Context windows are longer

Long-context and multi-turn systems generate large KV caches.

The cache may need to remain accessible across many requests, especially for agentic systems, enterprise assistants, retrieval-heavy applications and persistent sessions.

3. AI systems serve more concurrent users

Inference infrastructure must balance throughput, latency and cost across many sessions.

Keeping everything in HBM is fast but expensive. Moving everything to SSDs can save cost but adds latency and I/O overhead.

A high-capacity tier close to compute can therefore be valuable even if it does not match HBM latency.

HBF vs HBM vs DRAM vs SSD

HBF makes the most sense when viewed as part of a hierarchy rather than as a direct replacement for one existing technology.

HBF is not “flash as fast as HBM”

The bandwidth targets are impressive, but bandwidth and latency are not the same thing.

HBM is based on DRAM and optimized for extremely low-latency, highly parallel access.

HBF is based on NAND flash. Even with massive parallelism, advanced packaging and a very wide interface, flash retains fundamentally different access characteristics.

So a 3 TB/s peak HBF figure should not be interpreted as equivalent behavior to a 3 TB/s HBM subsystem.

The First HBF Specification

The specification introduced at FMS 2026 gives the clearest picture so far of what the technology is intended to become.

According to current reporting on the specification:

The specification also reportedly defines electrical/interface characteristics, packaging, reliability requirements and software I/O behavior.

An important caveat is that the complete specification was not yet broadly published on OCP’s public site as of August 7, so several implementation details remain visible mainly through vendor disclosures and technical reporting.

That means developers should treat the current numbers as standard targets, not as independently verified shipping-product performance.

How HBF Is Expected to Connect to Accelerators

The purpose of HBF is not simply to create a faster storage device attached through a conventional SSD path.

The architecture is intended to place flash much closer to the accelerator.

SK hynix has described UCIe as a mechanism for simplifying integration with heterogeneous compute platforms.

UCIe is designed for high-bandwidth die-to-die communication inside advanced packages. That opens several possible system designs:

• accelerator chiplets plus HBM plus HBF in one package;
• HBF attached through a dedicated base die;
• HBF connected to custom AI accelerators through a standardized chiplet interface;
• mixed HBM/HBF memory hierarchies where software moves data based on temperature and reuse.

Sandisk has also referred to an xPU-HBF interface. Public information does not yet make every relationship between that interface and UCIe completely clear, so it is safer to treat the exact link implementation as still evolving.

Why 512GB per Package Matters

Current HBM provides tremendous bandwidth, but capacity per stack remains relatively modest.

Tom’s Hardware notes that an HBM4 stack can reach around 64GB, while the first HBF specification targets up to 512GB per package.

That is roughly an 8× capacity difference per package at the stated maximums.

The architectural impact can be substantial.

Imagine an accelerator package with multiple HBF stacks:

Those are theoretical capacity totals, not announced commercial configurations.

But they illustrate why HBF is interesting: it creates a plausible path to terabytes of memory physically close to accelerator silicon without attempting to build all of that capacity from HBM.

What 3 TB/s Really Means

The highest disclosed HBF bandwidth class is approximately 3.0 TB/s.

That is an enormous number for a NAND-based memory tier.

However, it needs several qualifications.

It is a specification target

There is not yet a broad set of independent production benchmarks demonstrating sustained 3 TB/s HBF operation in commercial AI servers.

Peak bandwidth is not application throughput

Real throughput depends on:

• request size;
• random versus sequential access;
• queue depth;
• controller design;
• flash parallelism;
• software scheduling;
• thermal limits;
• error correction;
• write behavior;
• workload locality.

Latency may matter more than bandwidth

For some workloads, a massive sequential read bandwidth is extremely useful.

For others, frequent fine-grained random accesses can expose flash latency that HBM avoids.

HBF’s success will therefore depend heavily on how software maps models, cache state and tensors between HBM and HBF.

Where HBF Could Help AI Inference

1. Large model weight storage

Inference often reads model parameters far more frequently than it modifies them.

That read-heavy behavior maps naturally to persistent flash.

A large portion of a model could potentially reside in HBF while the hottest layers, activations or frequently reused data remain in HBM.

2. Mixture-of-Experts models

MoE models contain large numbers of total parameters but activate only a subset for each token.

That makes tiered memory especially interesting.

Frequently selected experts could remain in HBM while colder experts reside in HBF and are promoted when needed.

3. KV-cache storage

Long-running agents and large-context applications can accumulate substantial KV-cache state.

HBF could provide a higher-capacity near-compute tier for cache data that does not justify occupying premium HBM continuously.

4. Retrieval and embedding stores

Some inference systems repeatedly access large vector databases, embeddings or intermediate artifacts.

Keeping a much larger portion of that data near compute may reduce movement through conventional storage and host-memory paths.

5. Multi-model serving

AI servers increasingly host many models or adapters simultaneously.

A persistent HBF tier could potentially let systems keep more models immediately available without consuming all HBM capacity.

HBF and KV-Cache-Heavy Workloads

KV-cache growth is one of the clearest examples of why AI memory hierarchies are changing.

Long-context and multi-turn systems repeatedly reuse prior context.

Research on CXL-based hybrid memory, SSD-backed cache tiers and distributed KV-cache systems already shows that the industry is exploring ways to move less latency-sensitive cache data out of expensive accelerator memory.

HBF attacks the same economic problem from the hardware side: put far more persistent capacity closer to the accelerator itself.

A likely future architecture is therefore not HBF-only.

It is more likely to be:

HBM for hottest data → HBF for large reusable state → SSD/CXL tiers for colder data → remote object storage for bulk datasets.

Software will determine whether that hierarchy works well.

Could HBF Help Local AI?

Potentially, but not immediately.

The technology is currently aimed primarily at data-center, enterprise and accelerator-class systems.

There are no mainstream consumer GPUs with HBF available today.

However, the underlying concept is relevant to local AI because local inference faces exactly the same constraint: VRAM capacity is expensive.

If future workstation or edge accelerators eventually integrate HBF-like memory, they could potentially keep models far larger than normal VRAM capacities close to the compute device.

For example, a future accelerator containing:

• tens or hundreds of gigabytes of HBM/DRAM for active computation; and
• hundreds of gigabytes or terabytes of HBF for weights and colder state

could run workloads that today require multi-GPU systems or large system-memory offload.

But this remains a future architectural possibility, not a shipping consumer feature.

What HBF Does Not Solve

HBF is promising, but it does not eliminate the major engineering problems in AI infrastructure.

It does not replace HBM latency

Flash remains fundamentally different from DRAM.

Latency-sensitive tensors and hot activations will still benefit from HBM.

It does not automatically make inference faster

If software places the wrong data in HBF, page movement and cache misses may erase much of the bandwidth advantage.

It does not eliminate memory-management complexity

Tiered memory creates new policy questions:

• What stays in HBM?
• What moves to HBF?
• When should data migrate?
• How is cache eviction handled?
• How are multiple accelerators coordinated?

It does not eliminate NAND endurance concerns

Inference weight storage is often read-heavy, which is favorable for flash.

But workloads that write large amounts of rapidly changing cache state may raise endurance questions that cannot be answered until production HBF hardware is characterized.

It does not have established commercial pricing

Claims that HBF will be “cheap HBM” are premature.

Advanced packaging, base dies, specialized flash and high-speed interfaces all add cost.

The meaningful metric will be cost per useful gigabyte at a required bandwidth and latency, not raw NAND pricing.

Adoption and Ecosystem Risks

A memory standard matters only if accelerator vendors adopt it.

Sandisk and SK hynix have taken the important step of placing the work under the Open Compute Project rather than keeping it entirely proprietary.

But adoption is still early.

Google and Tenstorrent have publicly shown interest in the HBF effort, while several major accelerator and semiconductor vendors have not publicly committed to products using the standard.

That creates several risks.

Competing architectures

HBM capacity is also increasing.

CXL memory pooling, memory-semantic SSDs, processing-near-memory designs and custom accelerator packages all compete for parts of the same problem.

Software support

The most technically elegant memory tier can still fail if frameworks and runtimes cannot place data intelligently.

HBF will likely need support from:

• accelerator runtimes;
• operating systems;
• inference engines;
• compilers;
• memory managers;
• distributed-serving frameworks.

Vendor participation

An open standard becomes far more valuable when multiple memory manufacturers, accelerator designers and packaging vendors implement compatible devices.

That ecosystem is not yet mature.

Roadmap and Availability

The development timeline is moving quickly.

Those future dates are vendor targets, not guarantees.

Volume production schedules, accelerator partners and commercial pricing remain to be announced.

Frequently Asked Questions

Is HBF the same as HBM?

No. HBM uses DRAM and is optimized for extremely high bandwidth and low latency. HBF uses NAND flash and targets far higher capacity and persistence while accepting different latency characteristics.

Is HBF just an SSD in a different package?

No. Conventional SSDs connect through storage-oriented interfaces and sit much farther from the accelerator. HBF is intended as a near-accelerator memory tier with far greater parallelism and bandwidth.

How much capacity does HBF support?

The first disclosed specification targets packages of up to 512GB.

How fast is HBF?

The initial specification defines bandwidth classes from roughly 0.4 TB/s to 3.0 TB/s. These are specification targets; independent commercial-product benchmarks are still needed.

Does 3 TB/s make HBF faster than HBM4?

Not in every meaningful sense. Peak bandwidth may overlap or exceed the bandwidth of a single HBM4 stack in some comparisons, but HBM uses DRAM and should retain a major latency advantage. Comparing only headline TB/s figures is misleading.

Is HBF available to buy?

Not as a mainstream production memory product for consumers or general server builders as of August 7, 2026.

Will NVIDIA or AMD GPUs use HBF?

Neither company has publicly announced a shipping HBF-based GPU in the sources reviewed for this guide.

Is HBF an open standard?

Sandisk and SK hynix are developing HBF through an Open Compute Project workstream. The first specification has been disclosed, although the full specification was not yet broadly available through OCP’s public site when this guide was validated.

Could HBF replace GPU VRAM?

It is more realistic to expect HBF to complement HBM or other fast accelerator memory rather than completely replace it.

Final Assessment

High Bandwidth Flash is one of the more important memory developments to emerge from FMS 2026 because it targets a real structural problem in AI infrastructure: compute is scaling faster than affordable near-accelerator memory capacity.

The first specification makes the ambition concrete:

• up to 512GB per package;
• approximately 0.4–3.0 TB/s bandwidth classes;
• NAND-based persistence;
• open-standard development through OCP;
• packaging intended for direct integration with heterogeneous compute.

If vendors can deliver those properties at acceptable latency, endurance, power and cost, HBF could become a valuable middle tier between expensive HBM and conventional storage.

The most likely outcome is not “HBF replaces HBM.”

It is a more sophisticated hierarchy:

HBM for the hottest data, HBF for large near-compute model state, and SSD/CXL storage for colder capacity.

That architecture could materially change the economics of serving very large models, persistent agents and multi-model inference systems.

But the decisive evidence has not arrived yet.

The next things to watch are real silicon, accelerator design wins, sustained-performance benchmarks, latency measurements, endurance data, pricing, software support and production schedules.

Until those appear, HBF should be treated as a serious emerging memory standard with unusually strong potential—not as a proven replacement for today’s AI memory stack.

Sources

Primary and official sources

• Sandisk — Sandisk and SK hynix Begin Global Standardization of Next-Generation Memory Solution, High Bandwidth Flash (HBF), February 25, 2026: https://www.sandisk.com/company/newsroom/press-releases/2026/2026-02-25-sandisk-and-sk-hynix-begin-global-standardization-of-next-generation-memory-solution-high-bandwidth-flash-hbf
• SK hynix — SK hynix and Sandisk Begin Global Standardization of Next-Generation Memory ‘HBF’, February 2026: https://news.skhynix.com/en/sk-hynix-and-sandisk-begin-global-standardization-ofnext-generation-memory-hbf/
• Sandisk — FMS 2026, August 4–6, 2026: https://www.sandisk.com/company/newsroom/events/fms-2026
• Open Compute Project: https://www.opencompute.org/
• UCIe Consortium: https://www.uciexpress.org/

Independent technical reporting

• Tom’s Hardware — New HBF spec outlines tech that can give GPUs terabytes of extra memory, August 4, 2026: https://www.tomshardware.com/pc-components/ssds/sandisk-and-sk-hynix-unveil-spec-up-to-16-hi-nand-stacks-3-tb-s-bandwidth-ucie
• PC Gamer — NAND and HBM have been merged into a new high-bandwidth flash tier, August 2026: https://www.pcgamer.com/hardware/memory/nand-and-hbm-have-been-merged-into-a-dorky-megazord-for-datacenters-by-sandisk-and-sk-hynix/

Source-quality note: Capacity and bandwidth figures in this article describe the first disclosed HBF specification. They are not presented as independently measured shipping-product benchmarks. Where the complete public OCP specification or production silicon data was unavailable, the guide explicitly preserves that uncertainty.