The cost pressure in the AI Chip industry is shifting from "who has the most powerful logic chip" to "who can SECure enough high-bandwidth mEMOry (HBM) at a reasonable price."
According to estimates released by Epoch AI on May 21, the proportion of HBM memory in the total component spending for AI chips designed by NVIDIA, AMD, Google, and Amazon rose from 52% in Q1 2024 to 63% in Q4 2025. This data is production-weighted and is not officially disclosed by the manufACTurers.
The significance of these figures lies in bringing a frequently underestimated link in the AI computing competition to the forefront: while advanced process GPUs or ASICs remain critical, high-bandwidth memory has become the larger cost item in large model training and inference clusters. For cloud providers, capital expenditure pressure in 2026 may stem not just from buying more chips, but from the rising cost of memory within those very chips.
Why HBM Has Become the Largest Cost Item
The rising proportion of HBM is driven by fundamental changes in AI chip architecture. Large models require higher memory bandwidth to feed computing units. Consequently, chips like the Nvidia H100, H200, and B200, the AMD MI300 series, as well as Google TPUs and Amazon Trainium accelerators, are allocating a larger portion of their budget to high-bandwidth memory located close to the computing core.
Epoch AI's component scope includes HBM, 3nm to 5nm logic chips, TSMC CoWoS advanced packaging, and aUXiliary components like Substrates and power supplies. It is important to note that this discussion focuses on "AI chip component costs" and cannot be directly equated to the total cost of an entire accelerator card, server, or data center.
| Component | Q1 2024 Share | Q4 2025 Share | Analysis |
|---|---|---|---|
| HBM Memory | 52% | 63% | The largest cost item continues to expand. |
| Logic Chip | ~14% | ~13% | Share remains stable; does not imply absolute Cost Reduction. |
| Advanced PackAGIng | 19% | 15% | Share squeezed by HBM. |
| Auxiliary Components | 15% | ~10% | Relative weight deCLIning. |
The implication of this table is direct: within the AI chip cost structure, the tightest constraint is not necessarily the GPU logic die, but potentially the HBM stack and related supply ARRangements.
Component Spending More Than Doubles, Driven by HBM
In absolute terms, the change is even more pronounced. Epoch AI estimates that the total component spending for relevant AI chips by the four designers increased from APProximately $22 billion in 2024 to $52 billion in 2025. Of this, HBM spending surged from about $12 billion to $32 billion, contributing roughly $20 billion of the incremental growth.
Viewed horizontally, this explains why the HBM capacity and yield of SK Hynix, Samsung, and Micron have become keywords in cloud computing capital expenditure meetings. Previously, the market focused on NVIDIA GPU delivery times, TSMC's advanced processes, and CoWoS capacity. However, during the transition from H100 to H200 and B200, the impact of memory capacity and bandwidth on overall system performance has become more direct, thereby increasing HBM's pricing power.
This does not mean logic chips are unimportant. With the logic chip share remaining stable at around 13%, it remains a high-value segment. However, when HBM prices rise and supply is tight, the marginal cost for cloud providers to expand clusters is more easily driven by memory constraints.
Cloud Providers Must Watch Component Prices in 2026
The most directly affected parties are AI infrastructure procurement teams and cloud provider finance departments. Microsoft has noted in its outlook of approximately $190 billion in capital expenditures for fiscal year 2026 that about $25 billion is attributed to higher component prices. Similarly, Meta has raised its 2026 capital expenditure range by $10 billion, citing pressure from rising component costs.
This shift will alter procurement behaviors:
Locking in Supply: Cloud providers may lock in HBM supplies earlier and sign longer-term agreements with memory manufacturers.
Design Trade-offs: In-house chip teams will need to make finer trade-offs between performance, memory capacity, and packaging complexity.
Indirect Impact: For AI Startups and enterprise customers, the impact will be more indirect. If cloud service providers bear higher hardware costs, prices for certain training or inference services may not drop immediately, and discounts may become more selective based on client and usage volume.
There are boundaries to this estimate. HBM unit prices, logic die costs, CoWoS pricing, quarterly production volumes, and chip mixes will all affect the final proportions. For instance, the shipment structures of General-purpose accelerators from NVIDIA and AMD differ from the in-house chips of google and Amazon, and these weight changes affect overall results. What is certain, however, is the direction: if HBM remains in short supply in 2026, the constraints on AI computing expansion will be written more on memory supply sheets than on GPU press releases.
Comments & Questions (0)
No comments yet
Be the first to comment!