Qualcomm's High Bandwidth Compute Stacks LPDDR to Replace HBM4 in AI Workloads

Full text

Qualcomm introduces High Bandwidth Compute (HBC) memory architecture

It leverages a hybrid design stacking LPDDR memory in a 3D space, leveraging multiple layers to essentially replace what the current generation of High Bandwidth Memory (HBM4) does

The move, which makes use of Qualcomm's extensive experience with LPDDR, is not only power-efficient but also offers massive amounts of bandwidth and up to 768GB of stacked memory for AI workloads

Qualcomm is reigniting its Data Center ambitions, building on its expertise as a chip designer that excels in the low-power compute segment by focusing on an entirely new architecture: High Bandwidth Compute (HPC).

The solution is a hybrid take on existing LPDDR memory that Qualcomm has successfully stacked in 3D vertical space, not unlike the industry-standard High Bandwidth Memory (HBM) and its latest iteration, HBM4, while delivering significant power savings along the way.

The move is possible by Qualcomm offering a near-memory compute architecture that combines memory with a compute-based die, with the former stacked vertically on top of the latter, effectively enabling up to 133 TB/s.

An AI memory offering for the future?

While the current industry standard, HBM4, is already widely used, Qualcomm's promised offering is expected to appear by mid-2027 as part of its next-generation AI inference accelerator, the AI250.

HBC Gen 1 offers a theoretical 768GB of capacity that HBM4 struggles to match, and Qualcomm's published 133TB/s bandwidth is an achievement, given that modern HBM4 solutions offer approximately 3.3TB/s per stack at the higher end.

Some of these bandwidth claims, however, might be a bit of an unfair comparison, as while HBM4 delivers raw bandwidth, Qualcomm's solution (and its theoretical speeds) is possibly in play only because it performs much of the compute on-die, making for an apples-to-oranges comparison in some ways.

Qualcomm, however, scores important wins with an AI industry increasingly obsessed with power , or rather, the lack of it, to continue many of their planned buildouts by touting its efficiency wins where it claims anywhere between 6x bandwidth per watt versus HBM for larger batch sizes and as much as 200x efficiency gains when it comes to a mix of small and large inference batches, such as coding assistants.

Qualcomm's partner list includes Meta and Microsoft, with the former's multi-generational agreement to use Qualcomm’s processors for AI being highlighted as an important win. Microsoft's CEO, Satya Nadela, reassured investors by detailing the software giant's partnership with the chip designer across the PC, local AI, and data center segments.

Given that Microsoft is increasingly looking to address the environmental footprint of its AI data center rollout, with its CEO already reassuring concerned parties and communities that the Redmond-based tech giant aims to be mindful of the water and power footprints of both currently planned and future data centers, this makes efficiency an even more important theme of late.

Qualcomm's solution to 'eliminate the HBM tax,' however, does not exist in a void; Competing solutions such as High Bandwidth Flash , backed by Samsung, SanDisk and SK Hynix, are also shaping up as potential competitors that focus on a low-write, high-read situation that most AI inference workloads tend to be.

Perhaps more importantly, Qualcomm's solution and the impressive numbers it offers do not have any third-party's independent test results yet that could verify its efficiency claims, even as Microsoft's vote of confidence is seen as an important one for one of the most important players in the mobile SoC business as it gears up to take a share of a growing, but increasingly competitive datacenter pie in the coming decade.