The AI data center is entering a different phase
As enterprise moves from training models to running them continuously in production, infrastructure requirements are changing; the challenge is no longer simply maximising accelerator performance – operators increasingly need to balance compute, memory, networking, power and cooling across heterogeneous systems.
Intel is positioning itself around that shift, with its recent AI infrastructure activity putting particular emphasis on inference, orchestration and data movement, highlighting the architectural challenges of building purpose-built inference infrastructure and presenting heterogeneous computing, disaggregated inference and intelligent orchestration as important building blocks for scaling agentic AI.
Inference changes the data center equation
Training remains a major source of AI infrastructure demand, but production inference introduces a different operating profile. AI agents can generate sustained streams of inference requests while interacting with applications, databases, APIs and other services. That means the infrastructure supporting them cannot be optimised around accelerator throughput alone. Intel’s strategy is to put its Xeon CPUs at the centre of this broader system, handling orchestration, concurrency and data movement around accelerated workloads. The company’s Xeon 6+ platform is designed for scale-out environments and includes high memory bandwidth, PCIe Gen 5 and CXL connectivity, alongside workload-level energy telemetry. For data center operators, that is significant because utilisation increasingly becomes a whole-system issue.
Memory becomes a critical constraint
Intel are also targeting one of the less visible challenges created by larger AI models: memory capacity and bandwidth. It’s forthcoming Crescent Island data center GPU is designed specifically for inference and combines up to 480GB of LPDDR5X memory with a 350W air-cooled PCIe form factor. The company says the architecture is intended to handle large, token-intensive workloads while improving performance per watt. The air-cooled design is particularly interesting for operators. as well – not every AI workload necessarily needs to be deployed in the highest-density, liquid-cooled environment. If inference workloads can be served effectively using lower-power accelerator configurations, operators potentially gain more flexibility in where those workloads are deployed.
Networking is part of the AI system
Intel is also treating networking as a performance and efficiency issue rather than a separate infrastructure layer, it’s Ethernet E835 portfolio supports connectivity up to 200GbE and includes RDMA capabilities for reducing CPU overhead and improving data movement. Intel is positioning the technology for AI, cloud and other high-density environments – that reflects a broader reality for AI facilities: moving data between compute resources can become just as important as the raw processing capability of the accelerator.
The AI data centre becomes a system
The significance of Intel’s recent activity is less about any single chip. It points towards an AI data center architecture in which CPU orchestration, accelerator capacity, memory, networking and power efficiency have to be designed together.
For operators planning AI infrastructure, this means the question is increasingly not simply how many GPUs a facility can support – it is how efficiently the entire system can turn available power, cooling and compute capacity into useful AI work. As inference and agentic workloads grow, that system-level optimisation could become one of the defining infrastructure challenges of the next generation of AI data centers.
