Operators boost AI agent output within energy limits using AMD's 32,768-core CPU-only Helion rack design

AMD has released the Helion rack blueprint to address the specific needs of running AI agents rather than training large language models. According to The Next Web, this open specification focuses on efficiency, with partner systems expected to arrive in November. The primary value proposition is maximizing the number of agents that can operate within a given watt of power.

The design was introduced at the Open Compute Project summit in San Jose, The Register reported. AMD offers two configurations: a CPU-only version for hyperscale operators and a mixed CPU and GPU version for enterprises. The CPU-only rack is the headline feature, supporting up to 32,768 cores. This density is achieved through 128 sockets of AMD’s 256-core Venice processors, built on the Zen 6 architecture.

The blades are available in 400W dense or 600W balanced power profiles. In contrast, the mixed rack holds a maximum of 16 Instinct MI350P accelerators. These are distributed across two eight-GPU servers that AMD states run independently. There is no scale-up fabric connecting these components, meaning this setup is not designed for frontier model training. For training workloads, AMD continues to promote the Helios rack, which features 72 MI455X parts and 31 terabytes of HBM4 memory.

A distinct feature of the enterprise rack is a token router, a detail mentioned only in AMD’s own release. This component directs AI agents’ model calls either to frontier models hosted in the cloud or to open-source models running locally within the rack. Dan McNamara, who leads compute and enterprise AI at AMD, stated that the goal is to enable partners to use available power efficiently. This approach allows organisations to balance performance with energy constraints.

Matt Kimball of Moor Insights and Strategy noted that the objective is to pack as many agents as possible into every watt. This focus resonates with sovereign AI operators, particularly in Europe, where power availability is a critical bottleneck. Denmark’s grid operator paused new connections in March, facing a queue of 60GW of projects against a peak demand of roughly 7GW. The constraint extends beyond Denmark, with Brussels adopting a Cloud and AI Development Act in June to triple data centre capacity within seven years.

Despite these challenges, demand remains high. A survey of 300 data centre decision-makers in the UK, Ireland and the Nordics identified power availability as the leading obstacle to delivering sovereign capacity. However, the Helion blueprint is not yet complete. AMD has not published pricing, plans to contribute the specification to the Open Compute Project rather than having already done so, and has not named any European system builders as partners.

How does the hardware differ from training racks?

The Helion rack is distinct from AMD’s training-focused Helios rack. While Helios carries 72 newer MI455X accelerators and 31 terabytes of HBM4 for heavy training tasks, Helion is lighter on graphics. The mixed version holds only 16 Instinct MI350P accelerators across two independent servers. The CPU-only version is the primary offering, utilising 128 sockets of 256-core Venice chips to reach 32,768 cores, prioritising agent inference over model training.

Why is power efficiency a central feature?

Power constraints are driving the design of the Helion rack. According to Matt Kimball at Moor Insights and Strategy, the job is to pack as many agents as possible into every watt. This is particularly relevant for sovereign AI operators in regions with grid limitations. For instance, Denmark’s grid operator paused new connections in March due to a massive backlog of projects relative to peak demand.

What role does the token router play?

The enterprise rack includes a token router that manages model calls for AI agents. AMD’s release notes indicate this component can send requests either to frontier models in the cloud or to open-source models running within the rack. Dan McNamara said this allows for efficient use of available power by routing workloads appropriately, though this detail was absent from other coverage of the announcement.

When will systems be available and what is missing?

Partner systems based on the Helion blueprint are due in November. However, the product is not fully defined yet. AMD has not released pricing information. It also plans to contribute the specification to the Open Compute Project rather than having completed that step. Furthermore, no European system builders have been named as partners, leaving the supply chain details unclear.

Background: Large language models

Large language models are predictive engines trained on vast text corpora. They generate output by calculating the most likely next token based on context. They do not know truth, only probability. This explains hallucinations and the need for rigorous verification of all outputs.

Read the full guide: How Large Language Models Work: Mechanics, Limits, and Reality

What to do and how to stay safe: AMD

• Assess current data centre power capacity to determine if high-density CPU racks align with existing energy budgets.
• Monitor the Open Compute Project for the final specification contribution to understand integration requirements.
• Evaluate whether a token router architecture fits your existing cloud and on-premises model strategy.
• Track partner announcements in November for hardware availability and regional support details.

Step-by-step guide: AI Usage Policy Checklist for Secure Enterprise Deployment

General security guidance from the Sandbox Desk newsroom. It is not confirmed advice from the organisations named in this story.

Frequently asked questions

Does the Helion rack support training large language models?

No, the Helion rack is designed for running AI agents, not training models. AMD’s Helios rack remains the product for training workloads.

How many CPU cores does the Helion rack support?

The CPU-only version of the Helion rack supports up to 32,768 cores using AMD’s 256-core Venice chips.

Is the Helion rack available for purchase now?

No, partner systems are due in November. AMD has not yet published pricing or named specific European partners.