Arm Breaks New Ground With a Processor Built From Scratch for AI Systems

Arm has spent more than three decades providing the blueprints that other companies use to build chips, but the company has now stepped into territory it has never occupied before. The Arm AGI CPU is its own processor, designed in house and built specifically for data centers running the most demanding AI workloads. It is a significant departure from the role Arm has played throughout its entire history.

Customers have been clamoring for off-the-shelf solutions that can be easily installed in server racks and scaled up, and Arm has delivered. The end result is a processor that simply nails it in the middle of systems running AI agents 24/7, juggling thousands of computers while keeping everything working smoothly and responsively. Each Arm AGI CPU includes 136 high-performance Neoverse V3 cores. Even when every thread is cranked up for hours at a time, these cores continue to produce speed. Memory bandwidth averages six terabytes per second for each core, with latency of only 100ns. This combination ensures that data flows readily, thus no core is ever idle while waiting for data to arrive. And the power usage is only 300 watts per chip, which is well within the cooling capacity of most modern data centers.

The chip was designed with dense configurations in mind. In a standard air cooled rack you get over 8,000 cores spread across thirty blades, with two chips and dedicated memory per blade. Switch to liquid cooling at 200 kilowatts and that same rack footprint scales up to nearly 45,000 cores, and when you consider that large facilities stack hundreds of these racks together, the numbers add up quickly. Arm claims a fully loaded rack delivers more than double the throughput of the latest x86 based alternatives at the same power draw and physical footprint, thanks to stronger sustained performance per core and more efficient memory handling.

The chip is built on TSMC’s three nanometer process, squeezing maximum performance and efficiency out of every square millimeter of silicon. The architecture was kept deliberately clean, shedding the overhead that tends to accumulate in older designs. Everything about it is oriented around moving data as quickly as possible, keeping the accelerators fed, and maintaining smooth traffic between agents whether the workload involves planning, decision making, or real time action.

Meta came on board as Arm’s first partner and customer, with engineers from both companies working together since 2023 to shape the chip around the demands of Meta’s massive AI infrastructure. The processor now sits alongside Meta’s own accelerators, giving the company more flexibility in how it manages workloads at scale. Early systems from Supermicro, Lenovo, and ASRock Rack are already shipping, with broader availability expected in the second half of the year.