The AI infrastructure boom goes beyond GPUs – CPUs, memory and networking take over

Full text

For the past two years, the generative AI conversation has been dominated by one piece of hardware: the GPU.

GPUs supplied the parallel compute needed to train large language models, and their scarcity quickly became a proxy for AI readiness.

But that shorthand is now incomplete.

The next phase of enterprise AI will not be defined by accelerators alone.

It will be shaped by CPUs, memory bandwidth, cloud capacity, networking, and the workflow systems that allow AI to move from casual experimentation into daily business operations.

AI’s true economic impact will not come from model access; it will come from whether businesses can turn AI into reliable, cost-efficient operational capacity.

AI is Becoming an Infrastructure Problem

The first wave of generative AI adoption was largely experimental. Employees used standalone tools to draft emails, summarize documents, or write code. These ad-hoc use cases were useful, but they did not require companies to redesign how work actually gets done.

The next wave is different. As AI moves deeper into enterprise workflows, IT infrastructure requirements become exponentially more complex.

A customer service tool that drafts a response is simple. An AI system that reads account history, checks policy, updates a CRM , logs the interaction, and triggers a follow-up task is an entirely different beast. This system does not just need a powerful model; it requires compute orchestration, secure data access, software integrations, permissions, audit trails, and fallback logic.

This is where the GPU-centric view fails. While GPUs remain critical for heavy inference, CPUs coordinate how these workloads interact with databases , APIs, security layers, and operating systems. As a result, memory bandwidth, latency, and power availability are becoming the true strategic constraints.

The High Cost of Unstructured AI Usage

The early enterprise playbook was simple: give employees access to powerful tools and see what happens. While this accelerated learning, it also exposed a massive financial vulnerability. Individual, unstructured prompting is expensive, difficult to measure, and hard to tie to tangible business outcomes.

We are seeing a major corrective shift play out among tech giants. Microsoft recently began pulling back internal licenses for Anthropic's Claude Code—which was costing between $500 and $2,000 per engineer monthly due to high token consumption—and is forcing its Experiences and Devices division to transition to GitHub Copilot CLI ahead of its June 30 fiscal year-end.

Similarly, Uber completely exhausted its entire AI coding tools budget in just four months. The ride-hailing giant deployed Claude Code to roughly 5,000 engineers and aggressively stoked adoption using internal leaderboards. The experiment was incredibly effective—assisted systems generated nearly 70% of committed code—but token usage scaled faster than anyone anticipated, forcing Uber's leadership to publicly question the net ROI.

Consequently, the future of enterprise AI will move away from fragmented prompting toward a central intelligence model. Rather than thousands of disconnected interactions, companies will rely on shared intelligence layers—centralized systems that understand corporate data, apply consistent business rules, route tasks across applications, and track performance.

This model is inherently more efficient because the same intelligence is reused across workflows rather than recreated from scratch by individual users.

From Answers to Workflows

The most critical shift in enterprise tech is the transition from tools that answer questions to systems that perform work.

Traditional software is deterministic: a user clicks a button, and a system performs a known action. AI workflows are more dynamic. An agentic workflow can retrieve real-time data, reason through a multi-step process, interact with third-party software, and loop in a human for approval.

This puts immense pressure on the full technology stack. To unlock actual productivity gains, businesses need clean data infrastructure, disciplined governance, and robust integrations. Advanced models are useless if layered on top of fragmented, disconnected corporate systems.

Unprecedented Change Management and the "AI-Native" Workforce

As these agentic systems mature, the impact on global employment will trigger a corporate change management crisis on a scale never before seen. AI will fundamentally alter hiring patterns and role requirements long before it eliminates headcount at scale.

Historically, headcount was the default lever to scale capacity; more customers required more support staff. AI breaks that linear relationship. Instead of asking how many people are needed to handle an influx of volume, leaders will increasingly ask how much of a process can be handled by automated systems.

This environment will aggressively reward adaptability. Professionals who stay ahead of the technology curve, learn to design AI-enabled workflows, and manage systemic exceptions will disproportionately benefit.

Conversely, the risk of displacement is starkest for those relying purely on legacy industry experience. Traditional technical and managerial paradigms are being disrupted by a new cohort of AI-native developers , product managers, and team members. These professionals do not just use AI as an assistant; they build, manage, and think in terms of automated, model-driven systems.

Those who fail to transition from traditional operators to AI-native orchestrators risk being replaced by those who do.

AI Infrastructure is Economic Infrastructure

The broader economic impact of AI will be determined by how deeply it can be embedded into the core systems that run global businesses.

GPUs, CPUs, networking, and data centers form the physical foundation. Agent orchestration, security, and observability form the operational foundation. Together, they dictate whether AI remains a novelty or becomes a scalable business capability.

The GPU race was merely the opening chapter of the AI boom. The next chapter will be defined by the holistic compute, data, and workflow systems that allow AI to do real work at scale. That is the moment AI stops being a tool and truly becomes infrastructure.

Check out our list of the best IT automation software .

This article was produced as part of TechRadar Pro Perspectives , our channel to feature the best and brightest minds in the technology industry today.

The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit