AI networks must adapt to surging inference traffic demands

Full text

For the last few years, most AI infrastructure conversations revolved around the massive scale of AI training.

The AI training factories that leading AI companies have been building to develop frontier models and lead the AI race are nothing short of impressive: hundreds of thousands of GPUs, power consumption comparable to a medium city, and tens of petabits per second to scale training across multiple campuses.

But, as AI adoption accelerates and its applications expand, a larger infrastructure challenge is set to grab the spotlight. The new, real, and sustained pressure on networks is coming from the demand side of AI: inference.

Inference is what happens every time someone asks a chatbot a question, drops a file into a productivity tool for analysis, or relies on an AI-generated summary inside their inbox or search results. On a per-query basis, the traffic generated is lightweight, nothing comparable with a massive training run.

Now multiply that by the number of people interacting with AI across the globe – and consider that each time more it involves video, images, and supporting files – and you can see where the next major network demand driver is emerging.

With inference, the growth curve is steeper, global, and widely distributed, and it is now reshaping how data center connectivity is designed and the broader network ecosystem that connects cloud regions, metros and continents.

The inference inflection point

Even more powerful than users actively embracing AI faster than any technology before is the compound effect of AI features being injected into platforms that already serve billions.

Just take a look at your technology ecosystem and no doubt you’ve got AI capabilities now embedded in your search engine, email, office software , maps, social feeds and your smartphone. Embedding AI into those products is ensuring instant, global access and inevitable overwhelming usage.

The result is staggering growth in inference volumes. Take Google, for example, which reported the number of AI tokens it processes monthly increased 50 times year over year in early 2025… and then doubled it again just two months later and continues to grow at breathtaking pace, as seen on the 60 percent quarter-over-quarter token growth announced on April 2026.

Efficiency gains at the hardware and algorithmic level help attenuate the demand for additional resources, but nowhere near all of it. Delivering this increased inference volume requires the accelerated deployment of new GPU capacity and in a quickly growing number of inference data centers distributed across more geographies.

That geographic distribution is the first reason inference is not just a compute story but, at its heart, a networking story.

From text to video: multimodal models’ impact on the network

Until recently, AI's contribution to overall internet traffic has been limited. A text prompt and a text response amount to a few kilobytes — negligible next to a single minute of video streaming.

That’s changing quickly.

Multimodal models analyze and generate images, audio, video and 3D content. A user uploading a short HD video clip for analysis or editing pushes several megabytes upstream in seconds.

Researchers, students, and workers are increasingly pumping collections of documents into models to summarize key findings and generate detailed reports. Cloud-based video analytics that process camera feeds to provide insights and alerts are finding their way into viable business models.

Multiply those interactions by hundreds of millions to billions of users and inference traffic becomes a major driver of distributed and pervasive traffic flows.

Reasoning models add a second stressor. Rather than producing an instant response, they break problems into multiple internal steps, often pulling in supporting information in real time. A single user-visible answer can sit on top of dozens of background retrievals, sending megabytes of data between models, storage systems and external sources, much of it crossing data center boundaries.

And more pressure on the network comes from context window expansion. Frontier models can now ingest enormous prompts: think entire document sets, conversation histories, retrieved knowledge bases and more. Retrieval-augmented generation has become a widely adopted technique for many enterprise AI applications, and it involves injecting contextual knowledge into the model prompt on every query.

All told, these trends mean inference is no longer a lightweight workload from a network perspective. It is becoming a dominant driver of traffic growth, both between data centers and between users and the AI infrastructure.

How the DCI focus needs to shift

AI models are now distributed across regions, and usage signals and reinforced learning feedback must flow back to the centralized intelligence.

Multi-step and disaggregated inference workflows are increasingly spanning sites with complementary capabilities; think one for prefill focused on high-compute, large context processing, another for decode centered around low-latency token generation and memory and cache efficiency.

And sovereign AI requirements are pushing workloads into specific jurisdictions, multiplying the number of facilities that need to be tied together with high capacity and reliability.

Typical inference DCI links already operate at multiple terabits per second per route. The number of routes is growing alongside the capacity of each one, driven by more resilient and diversified interconnect topologies combined with a surge in the volume and geographic distributions of emerging inference-driven AI data centers.

Interconnecting inference data centers is just one side of the equation. The other involves connecting users, agents, things, and organizations to run their inference workloads across this meshed infrastructure.

Multicloud onramps are evolving to enable the movement of large enterprise datasets across AI platforms. More symmetric broadband access and scalable aggregation will be needed support widespread cameras to upload video for analysis on the AI cloud . The entire networking ecosystem needs to adapt.

So how do network operators keep up with demand and service providers take advantage of these trends?

Connecting the Dots – or Tokens

Network operators are already responding with planning centered on scalability, flexibility, efficiency, and security . To maximize capacity of every fiber pair—a resource that has never been more valuable and sought-after—they are adopting coherent optical platforms and systems capable of 1.6 Tb/s per wavelength.

Lumen Technologies, for example, is one such network provider that is building for tomorrow. It’s expanding its network at incredible speed, and to do so is leveraging a high-bandwidth 1.6 Tb/s coherent transceiver.

Additionally, the variable and diversified nature of inference traffic patterns make static networks less than ideal. Many operators are thus turning to AI-assisted multilayer network control to shift capacity and optimize performance in near real time.

Again, Lumen is a prime example of this; by leveraging a rich control suite the provider is gaining comprehensive visibility and maximizing usage and performance of its fiber assets from one point of control.

And to tie it all off, as inference traffic often includes sensitive data, network operators are baking in encryption at the optical layer within any new DCI deployment.

Inference workloads are evolving faster than any forecasting model can keep up. The networks best positioned for the next five years are the ones designed to scale in capacity, reach and intelligence without requiring major upgrades each time the workload spikes.

Training defined AI's first wave of infrastructure buildout. Inference is shaping its second – one much more globally distributed and intricate that will transform the networking landscape.

Use the best business cloud storage to manage your data .

This article was produced as part of TechRadar Pro Perspectives , our channel to feature the best and brightest minds in the technology industry today.

The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit