Arm Says CPU Architecture Is Key to Scaling Agentic AI Workloads

Full text

As AI systems continue to evolve and advance, it's not just data center architecture being affected, but the wider infrastructure industry as a whole.

As Agentic AI workloads continue to scale, it is the CPU layer being particularly affected, with efficiency, coordination, and cost per task become increasingly important defining factors.

We spoke to Mohamed Awad, EVP, Cloud AI at Arm, to find out more.

Gartner predicts that over 40% of agentic AI projects will be canceled by 2027 due to operationalization difficulties. How does Arm’s approach to "system-level" architecture specifically address the friction points, like cost and complexity, that are causing these cancellations?

I think the Gartner prediction reinforces something we’re hearing broadly across the industry. Most organizations aren’t struggling because today’s models aren’t capable enough. They’re struggling because operationalizing AI at scale is fundamentally different from demonstrating it in a pilot.

At the largest scale, we're seeing many of the same operational challenges play out in the physical world as well. Building AI infrastructure now means solving for power, supply chains, land, permitting, and deployment timelines just as much as advances in compute.

Moving from a handful of AI interactions to continuous, production-grade workflows requires a much more holistic view of the infrastructure stack – but it also requires organizations to rethink how AI interacts with their existing data and business processes. AI performs best where information is structured and workflows are well understood. As organizations extend AI into more complex enterprise environments, both the infrastructure and the operational environment have to evolve together.

That’s why we’ve focused on optimizing the system, from the CPU and platform through to the software ecosystem. As organizations move from pilots to production, success will increasingly depend on how efficiently their entire infrastructure works together – not just how capable any individual model or processor is.

We’ve seen companies like Uber exhaust their AI budgets in months due to unstructured prompting costs. How does moving intelligence from ad-hoc tools to a "central intelligence layer" anchored by Arm CPUs change the economic ROI for an enterprise?

I'd actually look at examples like Uber a little differently. Rather than signaling that AI is too expensive, they demonstrate how much value organizations are finding when AI is deployed successfully. The challenge is that demand is growing faster than the infrastructure needed to support it economically.

That's why we're seeing AI token economics improve at an extraordinary pace as models, silicon, software, and inference infrastructure become more efficient. At the same time, organizations are learning to redesign workflows rather than simply replacing existing processes with AI. Together, those trends are helping make AI more economical to deploy at scale.

Arm's role is helping drive that efficiency by optimizing AI infrastructure from the ground up. Improving performance, memory bandwidth utilization, and energy efficiency across the compute platform helps lower both capital and operating costs as organizations scale AI.

You’ve argued that AI’s true economic impact will come from turning AI into reliable operational capacity. How does Arm help businesses transition from "casual experimentation" to high-throughput, daily business operations?

The biggest shift happens when AI stops being a tool people occasionally use and becomes part of the operational fabric of the business. At that point, reliability, latency, security, and efficiency become just as important as model capability.

Our role is to provide the compute foundation that helps make that transition possible at scale. We’re part of the broader industry effort to optimize AI infrastructure across silicon, platforms, and software so organizations can deploy AI more efficiently and economically in everyday operations. Ultimately, the goal isn’t simply to automate work – it’s to make AI reliable enough to help people accomplish more as part of their everyday workflows.

While GPUs dominated the first wave of AI, the "agentic era" seems to be hitting a bottleneck at the coordination layer. Why is the CPU now the defining factor in whether an agentic workflow - which might check a CRM, update a database, and trigger a task - actually succeeds?

GPUs remain fundamental for training models and accelerating inference. What’s changing is everything that happens around those models.

Every time an agent retrieves enterprise data, coordinates multiple models, invokes external tools, or manages a business workflow, those activities depend heavily on the CPU. As organizations deploy more autonomous AI systems, those system-level workloads continue to grow, making the CPU an increasingly important part of delivering efficient AI infrastructure.

The question is no longer simply “How fast can I execute a model?” It’s “How efficiently can I execute an entire AI workflow?”

The new Arm AGI CPU claims more than 2x performance per rack compared with x86 platforms. What specific architectural choices were made in this "historic first" production silicon to handle the continuous, system-level workloads of agentic AI?

Agentic AI places fundamentally different demands on infrastructure than traditional cloud computing. Rather than optimizing for peak benchmark performance, we designed Arm AGI CPU around the realities of modern AI data centers – fixed power, cooling, rack density, and continuous system-level workloads. That means optimizing how data moves through the system – from compute and memory bandwidth to efficient bandwidth utilization – so the CPU can orchestrate the continuous data movement, coordination, and reasoning workloads that agentic AI creates.

The objective wasn’t simply to build a faster CPU. It was to help organizations deploy more useful AI within the physical and economic constraints that increasingly define AI infrastructure.

With Oracle Cloud joining the Arm AGI ecosystem and Google Cloud deploying Axion, we are seeing a massive shift toward custom Arm silicon. Does this signal the end of "general-purpose" data center architecture in favor of purpose-built agentic infrastructure?

I don’t think the story is custom silicon versus commercial platforms. What’s changing is that organizations increasingly want infrastructure optimized for the AI workloads they’re running, while maintaining the flexibility of a broad software ecosystem.

We’re seeing that play out in different ways across the Arm ecosystem. Many hyperscalers are building their own silicon while also deploying commercially available Arm-based platforms, including Arm AGI CPU. Both approaches reflect the same underlying trend: AI infrastructure is becoming more purpose-built, but customers still want the portability and ecosystem scale that comes from a common architecture.

Many agentic AI projects risk overtaking data center capacity. How does Arm’s emphasis on performance-per-watt allow organizations to scale agentic AI within existing air-cooled data centers without requiring massive new infrastructure builds?

Power has become one of the defining economic constraints of AI infrastructure. Organizations can’t assume they’ll always have another data center or another gigawatt available.

That changes how infrastructure is designed. Success increasingly depends on delivering more AI capability within existing power, cooling, and rack constraints. Performance per watt is no longer just an engineering metric – it’s becoming a business metric because it determines how much AI organizations can deploy without proportionally increasing infrastructure investment.

Industry reports suggest that AI will fundamentally alter role requirements, rewarding those who can transition from "operators" to "orchestrators". How is Arm’s hardware evolution supporting this shift in how work is actually redesigned?

As AI becomes embedded into everyday business operations, organizations increasingly need infrastructure that supports continuous coordination rather than isolated model execution.

Whether people are orchestrating AI agents or AI agents are orchestrating one another, the underlying infrastructure has to manage constant communication, context, and execution efficiently. That’s exactly the type of system-level workload we’re designing for. Ultimately, the goal isn’t to replace people – it’s to make AI reliable enough that people can spend less time coordinating work and more time creating value.

What is the risk to businesses and professionals who continue to rely on legacy industry paradigms while a new cohort of "AI-native" teams builds on top of agentic-ready infrastructure?

Organizations that continue treating AI as a standalone productivity tool may find it harder to capture the full operational benefits of agentic AI. The next phase isn’t simply about adopting more models – it’s about building infrastructure capable of supporting continuous AI operations across the business.

Organizations that begin building this capability early won’t just deploy AI sooner – they’ll learn faster. As teams begin redesigning workflows around AI, they build the experience needed to improve those systems over time. We’re already seeing that in areas like software development, where AI-native teams are moving well beyond simple code generation toward entirely new ways of building software.