Technology
Nvidia argues the memory wall needs a whole-stack fix, not more HBM
Nvidia says HBM capacity per GPU rose only about 3 percent between Blackwell Ultra and Vera Rubin while cost-adjusted agentic throughput rose up to 30 times on SemiAnalysis's AgentX benchmark, and that the gain comes from system design rather than memory volume. The levers it lists are spreading model weights across up to 72 GPUs, storing the KV cache in NVFP4 and pushing inactive context into cheaper storage tiers. The performance figures are vendor-reported and not independently verified.