APoV 003The CPU, and the CPU in the Agent Era

Ask a coding agent to fix a bug, and it will pause for a few seconds to think, decide to inspect three files, pause again, edit two lines of code, and run the test suite. When the test fails, it reads the traceback, rethinks, edits, and runs the suite again. Across a dozen such turns, several minutes pass.

Scenes like this play out dozens of times a day in a software engineer's routine. Break those minutes down, however, and the expensive GPU is working only during the brief seconds when the model is “thinking.” Reading files, installing dependencies, compiling code, running tests, spinning up a headless browser to check documentation—all the hands-on work falls on the CPU. For the rest of the time, both chips sit idle, waiting on network responses, disk I/O, or the output of the previous tool call.

Figure 1 captures only a single task in isolation. Inside a production data center, a GPU's idle gaps are immediately filled by other users' requests; it never truly sits idle. The sandbox running the agent's tools is another story: during the seconds the model spends thinking, the sandbox typically has nothing to do, yet it must stay resident in memory awaiting the next instruction.

In the chatbot era two years ago, this barely mattered. A conversation was a single question-and-answer turn, and the CPU was merely a stagehand feeding inputs to the GPU. Over the past six months, however, agents have crossed a threshold in handling long-horizon tasks. “Thinking” and “doing” now alternate across dozens or even hundreds of turns, putting the CPU onto the critical path for the first time. How fast it runs directly governs when the job finishes. The past half-year of industry headlines has revolved around this shift. On September 21, AMD's intraday market capitalization topped \$1 trillion for the first time. At Splunk's annual conference, Intel CEO Lip-Bu Tan said bluntly that the company can currently meet only about half of customer demand for its CPUs. NVIDIA has begun selling its in-house CPUs as standalone products. Even Arm, after 35 years of living off licensing fees, has stepped into the market to sell chips of its own.

Is this longtime supporting actor genuinely having a second act, or is it just the latest wave of market hype?

Butler, Laborer, and Sparring Partner

Start with what the CPU actually does in a modern AI system. Broadly speaking, it holds down three jobs: serving as a butler to the GPU, doing manual labor for agents, and acting as a sparring partner behind the scenes during model training.

Serving as the GPU's butler is its oldest job. Task scheduling and data movement fall under its care, as do batch scheduling, memory allocation, and prefix matching inside the inference engine itself. The SGLang team has estimated that in an unoptimized inference engine, these CPU overheads can eat up as much as half of total runtime. In long-context chats and agent workflows, the butler takes on an even heavier chore: staging KV caches on behalf of GPU memory. As a model reads through context, it leaves behind intermediate activation states; keeping them spares the model from recomputing prior tokens on the next turn. Whenever GPU high-bandwidth memory (HBM) fills up—or whenever an agent pauses to wait for an external tool to return—that state has to be offloaded into host CPU memory. The industry's favorite shorthand for this growing burden is the server “socket ratio” of CPUs to GPUs. On its first-quarter earnings call, Intel noted that while training systems typically pair one CPU with seven or eight GPUs, inference has tightened that ratio to one CPU for every three or four GPUs, and agentic workloads are pushing it toward 1:1. AMD CEO Lisa Su has offered a similar assessment.

The direction is right, but “sockets” is far too crude a yardstick. A 64-core chip and a 288-core chip both count as “1” on paper, and the sandboxes executing agent tools usually sit on separate clusters of general-purpose CPU servers anyway—NVIDIA itself now sells standalone racks of its Vera CPUs that never show up in the socket ratio of a GPU enclosure. To get the math right, you have to look at how many physical CPU cores and how much host memory actually sit behind each GPU (Figure 2). Across three generations of eight-GPU x86 servers—DGX H100, B200, and B300—the socket ratio has stayed flat at 1:4, with physical cores per GPU hovering between 14 and 16 for three years. The real overhaul happened in NVIDIA's full-rack systems built around its own CPUs. From the GB200 NVL72 to the Vera Rubin NVL72, the socket ratio stayed at 1:2 and physical cores per GPU edged up only from 36 to 44, but host memory per GPU jumped more than threefold, from 240 GB to roughly 750 GB. Core counts barely budged while system memory tripled—and the bulk of that extra capacity is there to hold offloaded KV caches.

The second job is manual labor for agents—the “doing” in our opening bug-fix scenario. Running code, compiling builds, executing test suites, spinning up headless browsers, and parsing PDFs or web pages all happen inside isolated sandboxes. Researchers at Georgia Tech and Intel profiled five representative agent workloads and found that in the heaviest category, tool execution accounted for roughly 90% of end-to-end latency. Yet the same paper also showed the flip side: in inference-heavy, tool-light workloads like Toolformer, model inference consumed about 88% of total runtime on a modest GPU, and still took 77% on an H200. You cannot make a blanket claim that agents are bottlenecked on the CPU; where the bottleneck falls depends on which tools the agent calls and how fast a GPU sits behind it.

The third job happens behind the scenes: acting as a sparring partner during model training. In reinforcement learning (RL), every step a model tries requires a live environment to compile code, run unit tests, and score the result. Data cleaning, deduplication, and synthetic data generation during pre-training are likewise massive consumers of CPU cycles, even if they sit outside this immediate loop. According to SemiAnalysis, several frontier AI labs, scrambling to secure enough compute for RL rollouts, have bypassed cloud providers altogether to buy x86 servers directly. Invisible to end users, this behind-the-scenes demand was likely the first catalyst to tighten general-purpose server CPU supply this year.

More revealing still, training and inference are starting to look like the exact same workload. When OpenAI unveiled o3 in early 2025, the novelty was that a model could run a quick Python snippet or web search mid-response to double-check its answer. Eighteen months later, that back-and-forth between thinking and doing has stretched into unattended runs lasting hours. In customer feedback shared by Anthropic during the September 22 launch of Claude Opus 5.5, one user handed the model an engineering task spanning six repositories and let it run overnight; it worked for more than 18 hours straight without losing the thread. A few days later, Google announced Gemini 4 Argon for select security partners—a model that autonomously discovers, verifies, and patches critical vulnerabilities, scoring 77.9% on the long-horizon software engineering benchmark DeepSWE. The orchestration harnesses wrapping these models are evolving in tandem. OpenAI unbundled the runtime behind Codex and ChatGPT into an Agents API complete with hosted sandboxes and automatic context compaction, while Claude Code Projects splits a single conversation into parallel cloud sessions that keep running after you close your laptop. The loop of “execute, score, rethink” that once existed only inside offline RL training has become the everyday routine of live inference. Once tasks stretch across hundreds of steps, the jobs of butler, laborer, and sparring partner braid into one.

The easiest mistake in sizing this shift, however, is crediting every second of an agent's wall-clock runtime to the CPU. I have seen plenty of commentary point to tool use and web browsing as proof that CPU demand is exploding—an argument that conflates active work with passive waiting. A process waiting for an external API response or a web page download is suspended, burning almost no CPU cycles at all. The Georgia Tech study offers a textbook contrast. In the first preprint of the paper, the chemistry agent ChemCrow was benchmarked on literature search, spending much of its 4-to-10-second runtime simply waiting on arXiv downloads. In the revised version, the authors switched the profiled tool to RDKit molecular conformation calculations; when computing large molecules, tool execution consumed 85% to 88% of runtime—genuine, compute-bound CPU load. Across the entire study, the workload with the highest tool share (roughly 90%) was exact vector search over a 115 GB local corpus. What actually burns CPU cycles are compilation, testing, browser rendering, vector retrieval, and RL grading. If you fail to strip out network wait times, your estimate of the CPU market will be inflated severalfold.

Still, just because waiting does not burn CPU cycles does not mean it is free. A suspended sandbox continues to tie up memory and network sockets. On the GPU side, the conversation's KV cache must either keep occupying pricey HBM or be evicted to host DRAM and SSDs until the tool returns. NVIDIA's Dynamo inference framework handles tool calls this way, and its documentation explicitly warns that a tool pause of just 2 to 30 seconds is enough for a conversation's prefix cache to be evicted from GPU memory, forcing the next turn to recompute it from scratch. The bill for waiting ends up landing squarely on memory.

From Laptops into Data Centers

Alongside longer task chains, a subtler shift played out over the past year: the “doing” changed address. Claude Code and Codex were the two breakout coding agents of the past year, and at first, almost everyone ran them locally—the model thought in the cloud, while compilation, test runs, and browser automation burned CPU cycles on the laptop sitting on your desk. The migration to the cloud started early: OpenAI launched cloud Codex in May 2025, running each task inside an isolated cloud sandbox, and Anthropic followed with a cloud version of Claude Code that October. By the second half of this year, nearly every major new agent product had moved execution into data centers, expanding from coding into team collaboration and everyday personal tasks. Meta's personal agent Muse, launched in September, gives every user a dedicated cloud virtual machine; Anthropic's revamped Claude Code Projects fans a single prompt out across parallel cloud sessions; and OpenAI's Dots, released in late September, assigns each digital persona its own cloud computer that keeps working long after you close the browser tab.

The reasons for moving are practical. Agent tasks now regularly run for hours, whereas laptops get closed, lose Wi-Fi, and get slipped into backpacks. Worse, an agent that spins up browsers, installs packages, and edits files on a local machine is fighting its user for the same screen and the same RAM. Moving execution to the cloud lets you check progress from a phone, lets teammates share a live environment, and contains any catastrophic crash inside a disposable virtual machine. Above all, it solves concurrency: when one person is directing five or six agents at once, no thin-and-light laptop can keep up. Claude Code Projects allows a single account to spin up 200 new threads a day, each backed by a full cloud session.

For server CPUs, this relocation matters far more than any benchmark score. When an agent ran locally, it burned compute on a machine the user had already bought and paid for, adding zero to a cloud provider's server budget. Once execution moves to the cloud, every compile pass and every browser tab turns into concrete data-center demand for vCPUs and DRAM. Take Claude Code's cloud sessions: each session runs on an isolated x86_64 Ubuntu virtual machine provisioned with roughly 4 vCPUs, 16 GB of RAM, and 30 GB of disk. Usage numbers from sandbox provider E2B show how steeply that curve has climbed: from roughly 40,000 monthly sandbox launches in March 2024 to more than 15 million in a single month in 2025, before crossing one billion cumulative launches in early June 2026. Part of this year's server CPU crunch comes from net-new agent workloads, but a meaningful slice is simply existing work that used to be scattered across millions of personal laptops and has now been consolidated into data centers. As for how many physical CPUs these sandboxes actually consume in aggregate, we will run that math in the market-sizing section below.

Once inside the data center, these sandboxes have settled into two distinct living arrangements. One works like checking into a hotel for a single night: spin up a clean virtual machine for a task and tear it down when the job is done, which is how Claude Code's standard cloud sessions operate. The other is a long-term lease: Muse keeps a dedicated virtual machine for each user, Grok Bot's personas share a persistent cloud computer, and each Dot maintains its own cloud environment while phone and desktop apps act merely as walkie-talkies. Personal agents built to manage daily life naturally gravitate toward the second model, because their work is cumulative—logged-in browser sessions, half-finished spreadsheets, and files from last week's conversation are far too cumbersome to rebuild from scratch every time. Persistent agents of this sort will only become more common.

Down at the infrastructure layer, however, the line between a one-off room and a long-term lease is much blurrier than it looks. Even if you leave a Claude Code session open, it automatically suspends after a few minutes of inactivity, keeping its files on disk and waking the same virtual machine when your next message arrives; threads in Claude Code Projects likewise pause their sandboxes between turns. Ultimately, “persistent” means preserving state, not leaving a CPU spinning around the clock. Grasping that distinction is the key to seeing which piece of hardware cloud agents actually push to the wall first.

Memory Hits the Wall First

Once sandboxes move into server racks, the ceiling on how many agents a physical host can pack is rarely set by CPU cores. The tax of virtualization isolation itself has been pared to the bone: minimalist microVMs like Firecracker add less than 5 MB of overhead per instance and boot in 125 milliseconds, while house-keeping chores like networking, storage, and hypervisor scheduling have long been offloaded to dedicated cards such as AWS Nitro, leaving nearly all of a host's CPU and RAM for tenant workloads. What eats up space is the baggage inside the sandbox—code repositories, language runtimes, dependency trees, plus a headless browser easily add up to hundreds of megabytes or several gigabytes. CPU cores are easy to oversubscribe, slicing a single physical core across a dozen idle sandboxes; memory is far less forgiving. Every loaded dependency and open browser page that has not been freed occupies physical RAM byte for byte.

Even products marketed as “an always-on personal cloud computer for everyone” never actually dedicate compute around the clock. Take E2B: when a sandbox pauses, it snapshots both the file system and memory state to disk, then stitches them back together on wake-up. While asleep, the sandbox bills at storage rates rather than compute rates, a model shared by most sandbox providers. Yet “pause” does not mean the same thing everywhere. E2B performs a full snapshot that immediately frees RAM for other tenants; Claude Code's cloud sessions also pause after a few idle minutes while retaining files, but Anthropic's documentation does not specify whether system memory is released at the same time. That implementation gap directly determines how many sandboxes a single host can stack.

Done thoroughly, snapshotting unlocks a second layer of memory savings. When Firecracker restores a microVM from a snapshot, it does not blindly read the entire memory image back into physical RAM upfront. Instead, it faults pages in from disk on demand and copies a page only when a sandbox actually modifies it (copy-on-write). As a result, a hundred microVMs spun up from the same base snapshot share a single set of physical memory pages for their read-only runtimes and libraries. The main catch is security: clone the same snapshot a hundred times and every instance wakes up with the exact same random-number pool and machine identifiers, so the hypervisor must immediately reseed entropy upon restore.

And memory happens to be the tightest bottleneck of 2026. In the first half of the year, server CPUs ran short first, holding up full rack builds and leaving several US cloud providers sitting on temporary stockpiles of DRAM; some buyers even stepped down server memory configurations from 96 GB and 128 GB modules to 32 GB and 64 GB sticks to trim costs and match the number of CPUs they could actually get their hands on. That breather lasted barely half a quarter. Server DRAM contract prices are projected to jump another 13% to 18% quarter-over-quarter in the third quarter; by the fourth quarter, as CPU deliveries loosened slightly, cloud providers rushed back into the market to buy general-purpose server memory, explicitly citing agentic AI as the driver. According to TrendForce's preliminary 2027 supply projections, bit output for server memory modules (RDIMMs) will grow just 15% to 20%, still lagging behind server CPU shipment growth. CPUs and memory go into the pot together; a shortage of either one holds up the whole dish.

When a motherboard's DIMM slots are maxed out, one pressure valve is attaching external memory over Compute Express Link (CXL), which hangs an expansion pool of RAM off high-speed interconnect lanes beyond the CPU's native DDR5 channels. In November 2025, Microsoft Azure began private previews of the industry's first cloud virtual machines backed by CXL memory expansion, which controller chipmaker Astera Labs says can boost per-server memory capacity by more than 1.5x. CXL does carry higher latency than direct-attached DDR5, but for parking sandboxes that spend 90% of their lives asleep, that extra latency is a rounding error.

GPUs Are Moving Too Fast

Everything discussed so far has been about volume—longer task chains, more sandboxes, larger RL training fleets—which governs how many total CPUs the market needs to buy. Beneath the surface, however, another force is at work in speed, dictating which kinds of CPUs command a premium. That force is Amdahl's Law, the oldest truism in computer architecture: if one part of a job cannot be sped up, making the rest arbitrarily fast will eventually leave total runtime hostage to the slow part.

Run a simplified thought experiment. Suppose a single agent step starts out taking 8 seconds of model “thinking” and 2 seconds of tool “doing.” If GPU compute doubles every generation while those 2 seconds of compilation and unit testing—riddled with serial dependencies and branches—speed up by just 25% per generation on a single CPU core, then across three hardware generations, tool execution swells from one-fifth of the step to roughly half (Figure 3). To be sure, this holds the amount of thinking constant: in practice, reasoning models are also generating more thinking tokens per turn, eating back part of the GPU's speedup. Single-thread speed is therefore more of a slow-moving structural variable, while concurrency volume remains the main engine behind this year's buying spree. Even so, the Georgia Tech paper offers a clean cross-sectional check: when the authors moved the exact same agents onto machines with faster GPUs, the end-to-end bottleneck shifted noticeably toward the CPU.

Why are the two chips improving at such wildly different speeds? Look first at the GPU. Large-model inference is built on matrix multiplication—thousands of independent multiply-accumulate operations where every additional compute unit on the die translates almost directly into higher throughput. The leap from NVIDIA's H100 to B200 is a textbook case of brute-force scaling: built on closely related TSMC 4N and 4NP processes, the B200 stitches two dies together in a single package and cranks up the power budget, pushing transistor counts from 80 billion to 208 billion. A compile pass or a unit test cannot use that kind of die-stitching at all. Better yet, GPUs have several levers a CPU simply cannot pull. Neural networks tolerate coarse numerical precision, letting chipmakers step formats down from FP16 to FP8 and FP4—roughly doubling throughput each time bit-width is halved—whereas no compiler or test runner can execute in 4-bit arithmetic. Dedicated Tensor Cores and on-package HBM stacks get a major overhaul every generation, and NVLink can lash 72 GPUs into a single logical machine. Measured on dense FP8 compute without sparsity tricks, the B200 delivers roughly 2.3x the compute of an H100 released two years earlier, and switching to FP4 doubles that again. That second doubling comes from changing numerical formats rather than apples-to-apples silicon gains, but to an agent, it still cuts “thinking” time in half.

A single CPU core answers to a very different set of physical laws. Compilation and unit testing are long chains of tightly coupled instructions, thick with conditional branches. Microarchitects can only squeeze a bit more parallelism out of a serial instruction stream through branch prediction and out-of-order execution, and that well has been running dry for years. In their classic computer architecture textbook, Turing Award laureates John Hennessy and David Patterson ran the numbers: to keep wasted work from mispredicted branches under 10%, a processor has to guess right on 99.3% of every branch—a bar very few general-purpose programs can clear. Clock speed hit a wall long ago as well: Dennard scaling slowed visibly after 2007 and effectively died around 2012, meaning higher frequencies slam straight into the power wall. As a result, annual gains in single-thread performance fell from roughly 50% in the early years to low single digits by the late 2010s. Over the past few years, AMD has clawed back some momentum across its two-year Zen 3, Zen 4, and Zen 5 cadence, lifting instructions per clock (IPC) by 19%, 13%, and 16% respectively—roughly 10% a year, a welcome rebound, yet still an order of magnitude behind a GPU's pace of more than doubling every two years. Extra transistors can only go toward packing in more cores, which lets one server run dozens more sandboxes at once, but does almost nothing to make a single incremental build finish several times faster.

The upshot is simple: every time you buy a faster GPU, you have to pair it with a beefier CPU and a much larger pool of memory, or the time saved by the new GPU gets given right back while waiting on tools. NVIDIA is simultaneously pairing each GPU with larger cores and triple the host memory while selling standalone racks packed with 256 Vera CPUs; Intel's reference rack at Computex crammed 36,864 cores into a single 100-kilowatt cabinet. Both are patching the exact same bottleneck.

To make matters worse, production agents often fire off several tool calls in parallel within a single step, yet must wait for the last one to return before starting the next round of reasoning. How long the step stalls is dictated by the slowest tool: one sluggish dependency resolution or one cold container start holds up the entire chain. For machines running sandboxes, strong average throughput is not enough; tail latency cannot be allowed to slip.

How Big Is the Business?

Back to those exuberant headlines at the start: are agent sandboxes really what bought out the server CPU market? A back-of-the-envelope calculation helps put the numbers in perspective. According to NVIDIA's own figures, a standalone CPU rack loaded with 256 Vera chips can run more than 22,500 isolated environments concurrently—roughly 88 per CPU, or about one per core. At that density, even if one million agent sandboxes were running simultaneously worldwide, they would need only about 45 racks, or 11,000 CPUs; scale that up tenfold to 10 million concurrent sandboxes, and it still comes to roughly 110,000 chips. At \$10,000 to \$20,000 for a flagship server processor (Intel's top-end Xeon 6980P carries a recommended price of about \$14,000), 10 million concurrent sandboxes translate into roughly \$1.1 billion to \$2.3 billion in chip purchases. Set against a global server CPU market of \$26 billion to \$29 billion in 2025, that is a single-digit percentage—and that is the one-time capital cost to build the fleet. Amortized across a four-to-five-year server depreciation cycle, the annual spend is smaller still.

This is, of course, a very rough sketch. No one discloses how many sandboxes are actually online at any given moment, and real-world deployments rarely pack instances as tightly as NVIDIA's reference math assumes—Claude Code caps a single session at 4 vCPUs. Yet even if you haircut the density, the order of magnitude does not change: online agent sandboxes alone come nowhere near explaining a \$20 billion to \$30 billion server CPU boom in 2026. As for the servers frontier labs are snapping up for RL training, an RL environment is just a sandbox under another name; measured by the same yardstick, its unit volume is similarly bounded.

So where did all the extra money actually go? Peel back the major chipmakers' earnings reports, and three other forces are doing the heavy lifting: host CPUs shipped inside GPU server racks; a routine five-year replacement cycle for aging servers installed during the 2019–2021 cloud buildout; and, most directly, across-the-board price hikes driven by tight supply. Live agent sandboxes are indeed the fastest-growing source of new demand, but the biggest line item on this year's ledger is simply that chips got much more expensive.

Intel's quarterly filings spell this out most plainly. Second-quarter revenue hit \$16.1 billion, up 25% year-over-year for its fastest growth in 15 years, with its Data Center and AI (DCAI) group jumping 59% to \$6.3 billion. Open the detailed tables in its Form 10-Q filed with the SEC, however, and server chip unit volume was up just 9%—with no way to tell how much of that was routine fleet refresh—while average selling prices (ASPs) soared 48%. The first quarter was even starker: unit shipments fell 5% year-over-year, and revenue growth was carried entirely by higher prices. How did ASPs climb so steeply? Half came from customers shifting toward higher-core-count parts, and the other half came from outright price increases: the recommended price on Intel's flagship Xeon 6980P reportedly jumped from \$12,460 to \$13,955. Add in the filing's explicit acknowledgment of bottlenecks in packaging substrates and memory, and Lip-Bu Tan's line about meeting “only half of demand” reads less like runaway demand alone and just as much like a supply chain dropping the ball.

The player actually feasting on incumbent market share is AMD. Second-quarter data-center revenue more than doubled year-over-year to \$6.7 billion (a figure that includes Instinct GPUs, as AMD does not break out CPU revenue separately). Even more striking was its guidance: server CPU revenue is projected to grow more than 80% year-over-year in the second half of this year, and more than 70% again in 2027. By Mercury Research's count, AMD ships roughly one-third of x86 server processors yet takes home nearly half of the revenue. That double-digit gap between unit share and revenue share is the premium AMD's high-core-count, high-ASP EPYC chips are commanding in the richest tier of the market.

Outside the x86 duopoly, cloud providers' own Arm chips are quietly carving off a massive slice of the pie. More than half of the new CPU capacity AWS has added over the past three years runs on its in-house Graviton chips, and Graviton5, which reached general availability in June, packs 192 cores onto a single socket; Microsoft and Google are pushing ahead with Cobalt and Axion as well. Arm claims its architecture now accounts for roughly half of CPU compute across the leading hyperscalers—though it has never spelled out whether that “half” is measured by cores, sockets, or some weighted compute metric, and no independent data exists to cross-check it, so it is best treated as directional.

Even players from adjacent lanes have crowded onto the table. NVIDIA's Grace CPU has topped \$5 billion in trailing twelve-month revenue, and its next-generation Vera is being unbundled from GPU racks for standalone sale. On NVIDIA's May earnings call, Jensen Huang said the standalone CPU business had nearly \$20 billion in “visibility”—a term referring to the demand pipeline visible from current customer inquiries, though plenty of Chinese and English media outlets flatly misreported it as “\$20 billion already sold,” confusing inquiries with booked revenue. Meanwhile, Arm unveiled its first self-designed, self-sold processor in March: the AGI CPU, packing 136 Neoverse V3 cores on TSMC's 3nm node within a 300-watt envelope, with Meta as its launch customer and OpenAI and Cloudflare on the roster. After 35 years of making a living selling blueprints, Arm finally could not resist stepping onto the field to compete directly with its own licensees.

As for how large this table can grow by 2030, the most telling subplot is how AMD has revised its own forecast three times in eight months: last November it put the 2030 server CPU market at roughly \$60 billion, doubled that to more than \$120 billion in May, and then hiked it to \$220 billion at its Advancing AI event in July. Working backward from the compound annual growth rates AMD gave each time, I found that all three projections trace back to the exact same 2025 baseline in the mid-\$20 billion range (Figure 6). The starting line underfoot never moved; only the slope of the line drawn into the future got steeper.

What is more, the “market size” numbers vendors throw around are not even measuring the same thing: NVIDIA says Vera opens up a \$200 billion “entirely new” market; Arm pegs the 2030 agentic data-center CPU opportunity at roughly \$100 billion; and AMD executives at an investor session even floated a \$280 billion figure covering complete “CPU server” systems. The same phrase is being used to package very different scopes that can neither be added together nor compared head-to-head; they are better read as turf-staking declarations. For perspective, even if you take AMD's most aggressive \$220 billion forecast at face value, the 2030 TAM it projected for AI accelerators at that very same event was \$1.4 trillion. The CPU has indeed graduated from a sidekick into a major business, but inside the broader AI compute pie, it remains a widening tributary.

The current balance of power is straightforward. The x86 incumbents still hold the lion's share of existing revenue, but the richest slice of incremental growth is being carved up among hyperscalers' in-house silicon, NVIDIA's bundled racks, and an Arm now selling chips of its own.

Who Is This CPU Built For?

Where the money flows, chip architecture follows. Trace the breakdown above one step further, and today's server CPU is splitting into two distinct species: the host socket swallowed inside a GPU rack to serve as the accelerator's personal butler, and the agent execution pool sitting outside as general-purpose cloud compute, running thousands of sandboxes and RL environments. For the past decade, the industry's defining argument was x86 versus Arm. That debate is still alive, but a more urgent question has overtaken it: who is this CPU actually built for?

The most radical transformation is happening in the host socket. Inside a Vera Rubin rack, Vera and Rubin are lashed together over a 1.8 TB per second NVLink-C2C interconnect into a single cache-coherent memory space; at that point, the CPU looks less like an independent computer and more like a massive off-chip cache and scheduler bolted onto the GPU. Even the x86 camp is having to bend to this template. One condition of NVIDIA's \$5 billion investment in Intel was that Intel build custom Xeon processors with native NVLink interfaces. AMD's Verano (EPYC 9006 LP), slated to ship in the second half of 2027, is even more explicit about being a dedicated GPU host: it drops standard server DDR5 altogether in favor of 24-channel mobile LPDDR5X memory and hooks directly to GPUs over AMD's proprietary xGMI fabric. The recipe is cribbed straight from Vera—trading DDR5 for LPDDR5X to get bandwidth and power efficiency, and using a proprietary high-speed link to bind tightly to the GPU.

Once the industry goes down that road, the host CPU ceases to be a standalone component that buyers can comparison-shop on its own. Whichever full-rack GPU system a customer signs up for dictates the host chip inside: buy a Vera Rubin NVL72 rack and you get Vera; buy an AMD Helios rack and you get Venice. Where x86 still holds its ground is the eight-GPU chassis tier—NVIDIA's DGX Rubin NVL8, for instance, still pairs its GPUs with Intel Xeon 6 processors. Yet that is precisely Intel's most uncomfortable bind: without a blockbuster full-rack GPU system of its own to pull its CPUs along, it is left hitching rides on other vendors' platforms at the very top of the market.

Whether the host socket becomes permanent captive territory for GPU makers comes down to who owns the wire connecting the CPU and the accelerator. In 2025, NVIDIA rolled out NVLink Fusion, opening NVLink to third-party CPUs and accelerators; Qualcomm and Fujitsu have already announced plans to build it into their server chips, though on the condition that the resulting systems be bought alongside NVIDIA silicon. On the other side, heavyweights including AMD, Intel, Google, Meta, Microsoft, and AWS banded together to form the UALink Consortium, releasing their 1.0 specification in April 2025 to link up to 1,024 accelerators in a cluster. If proprietary interconnects win out, the host socket stays lashed to the GPU rack; only if open interconnects gain real traction in data centers will cloud providers keep the freedom to mix and match CPUs and accelerators at will.

Fortunately, the host socket is only half the sky. Across the data-center network, the agent execution pools running sandboxes and RL environments remain general-purpose servers that cloud providers buy à la carte and scale independently. In that pool, Intel can lean on the core density of Xeon 6+ and its own fabs without having to hitch a ride with anyone.

Inside the execution pool, however, chipmakers have split along a different fault line: should you stack fewer, brawnier cores, or pack in as many small cores as possible? Wearing the same “2026 flagship server CPU” badge, the chips arriving this year look like two different product categories. Intel's Xeon 6+ crams 288 efficiency cores onto a socket and drops hyperthreading altogether; AMD's Venice, shipping in the second half of the year, splits into a dense variant with up to 256 cores and a high-frequency variant that trims the count to 96 cores while pushing clocks past 5 GHz; NVIDIA's Vera, also shipping in the second half, has just 88 physical cores, but each is a heavy 10-wide-decode core backed by 1.2 TB per second of LPDDR5X memory bandwidth. Plot core counts against theoretical memory bandwidth per core on the same chart, and the two design philosophies jump off the page (Figure 7).

Those two philosophies map neatly onto the two workloads discussed earlier. High bandwidth per core in Figure 7 feeds butler duties like shuttling data and catching offloaded KV caches. Out in the execution pool, a single incremental compile or unit test sitting on the critical path wants high single-core clock speeds, large caches, and low memory latency; meanwhile, thousands of sandboxes spending 90% of their time waiting on results care about how many instances you can pack per watt and per dollar. When Intel showed off two reference racks side by side at Computex—one tuned for low latency, the other for maximum density—it was drawing this exact dividing line. Both tracks are likely to coexist for years, splitting data-center racks into two distinct configurations depending on the job.

Behind all this activity sits an absurd reality: to this day, the industry does not have a single agreed-upon “agent benchmark.” AMD ran its own handpicked workload and declared that an EPYC rack delivers 2.37x the throughput of Vera; CoreWeave measured sandbox cold-start speed instead and concluded that Vera is more than 3x faster than x86. Everyone is measuring the market with whatever yardstick lets them win. If I were to draft a benchmark that actually meant something, it would lock down at least three numbers: a fixed per-sandbox spec (say, 4 vCPUs and 16 GB of RAM, matching Claude Code), the time in seconds to cold-boot an instance versus waking it from a snapshot, and 99th-percentile tail latency across tool calls—because as we saw earlier, an agent step only moves as fast as its slowest tool. Whoever turns that yardstick into the industry default will hold the power to define the next generation of server procurement.

As for the decade-old x86-versus-Arm debate, the agent era splits the answer across two types of fleets. The binary-compatibility moat x86 spent decades digging has certainly grown shallower, but it is far from filled in. Inside the internal fleets controlled by cloud providers and model labs—running inference schedulers and first-party services written in-house—recompiling the stack for Arm is straightforward, and the contest comes down to performance per watt and who can deliver on time. Once you step into sandboxes meant to reproduce messy external environments, however—whether a user's coding sandbox or a lab's RL training ground—countless prebuilt pip wheels, native npm modules, and legacy Docker images still assume x86 by default. Claude Code's cloud sessions run exclusively on x86_64 virtual machines to this day, with Anthropic's docs explicitly advising users to favor prebuilt x86_64 binaries, and the servers frontier labs are buying up for RL are likewise x86 machines. Arm has won far more tickets to the dance in this cycle than it ever did in the traditional SaaS era, but those tickets are mostly for the back kitchen that vendors control themselves—not yet for the wilds of user-facing sandboxes.

Pushed by Arm's advance, the x86 camp did something rare: in October 2024, Intel and AMD set aside decades of rivalry to form the x86 Ecosystem Advisory Group. Over the past year, they have moved briskly to unify AVX10 vector instructions, ACE matrix extensions, and the FRED interrupt model, while adding ChkTag memory-safety tagging—a feature Arm has had for years—to clean up fragmentation across their shared instruction set. Further out on the horizon, RISC-V notched two notable milestones this year: NVIDIA took the stage with SiFive at Hot Chips 2026 to lay out hardware standards for running CUDA with RISC-V host CPUs, and Alibaba unveiled its server-grade XuanTie C950. For the next two to three years, however, RISC-V's main proving grounds will likely remain China's domestic substitution market and embedded controller cores inside accelerators, rather than the general-purpose execution pools of major clouds.

Zooming back in to the next year or two, market share will be decided less by who designs the cleverest microarchitecture than by who can actually get wafers out of the fab. AMD's Venice uses TSMC's 2nm node, while AWS's Graviton5 and Arm's AGI CPU use 3nm—all of them fighting higher-margin GPUs and flagship smartphone chips for the same leading-edge lines. By SemiAnalysis's estimate, AI chips alone will swallow 86% of TSMC's N3 capacity by 2027. Intel's biggest trump card is owning its own fabs, with Xeon 6+ running on its internal 18A line; the awkward catch is that Intel can currently satisfy only half of its own customers' demand, and its worst bottlenecks are on mature volume nodes. Intel 3 capacity for its high-volume Granite Rapids chips is tight, packaging substrates are tighter still, and Intel is having to divert wafer allocations from other product lines to feed data centers. As for its next-generation 14A node, Intel's official timeline puts internal risk production in the second half of 2027 and volume ramp in 2028; even if risk production is pulled forward to the first quarter of 2027, as industry rumors suggest, distant water will not quench this cycle's thirst. In this window, buyer logic is brutally simple: buy whatever silicon is in stock first, and rewrite the software later.

What Comes Next?

By now, the question we started with largely answers itself. This CPU revival is real, not manufactured hype—but it is not the “changing of the guard” that much of the press has made it out to be. GPUs still command the lion's share of compute spending. What changed is that as agents stretched out the “doing,” fanned out concurrency, and moved execution into the cloud—while GPUs themselves kept racing ahead—the CPU and memory that once sat quietly backstage were yanked onto the critical path together.

Plenty of numbers on this ledger are still hanging in the air. What I most want to see is a cloud provider or research group start reporting how many CPU-seconds and how many gigabyte-hours of memory it actually takes to complete a real-world agent task. Until then, slogans like “CPU-to-GPU ratios heading to 1:1” or “agents needing 4x the cores” are fine as directional color, but useless for sizing a market. Equally unproven are Arm's claim of holding “half of compute at top hyperscalers” and Jensen Huang's \$20 billion in standalone CPU pipeline visibility. Whether those numbers mark a lasting structural shift or optimistic extrapolation at the peak of a shortage will become clear over the next few quarters of shipment data and earnings releases.

On the technical side, the biggest open question sits in the sandbox layer: will snapshotting and suspend-to-disk become the default foundation for every cloud agent? If they do, the vision of “an always-on cloud computer for everyone” will mainly drive cold storage and burst scheduling on wake-up, saving a massive slice of always-on CPU and RAM. If they do not, the memory wall will cap single-server density long before CPU cores run out.

Finally, keep an eye on the price cycle. The number-one driver behind this year's surge in server CPU revenue is higher prices, and during an industry-wide shortage, downstream buyers routinely place panic double-orders. With Intel ramping 18A and steering fab capacity toward data centers, and AMD shipping Venice in waves starting in the fourth quarter, supply should begin catching up by 2027. When lead times shorten and pricing softens, reported revenue growth will likely come down much faster than actual underlying compute demand.

For the past three years, tallying up AI compute meant counting how many GPUs were sitting in the data center. For the next few years, the accounting has to change: alongside GPUs, you have to count how much memory sits behind every card, how many cores sit in the execution pool, and how much space all those sleeping sandboxes are taking up across disk and RAM.