NVIDIA Spectrum-X Ethernet Photonics Now in Full Production, Builds Cleaner Path for AI Factories

AI factories keep getting bigger. Rows of servers packed with GPUs need to pass enormous amounts of data back and forth without slowing down or overheating the building. For years the network that moves all that data has relied on separate optical modules plugged into the face of each switch. Those modules convert electrical signals into light, send the light through fiber, then convert it back again. They work, but they draw a lot of power, add heat, and introduce more points that can fail.

NVIDIA’s Spectrum-X Ethernet Photonics stands out from the competition. The optical engines are now neatly bundled with the switch silicon. Light generation and conversion occur immediately next to the chip that determines where each packet goes. The end result is a network that is cleaner, cooler, and far more reliable, specifically suited for the scale of modern AI training and inference clusters. At the center of the system is a multi-chip module that houses hundreds of silicon photonics engines in parallel. Each one handles multiple high-speed lanes and converts electrical signals from the switch ASIC into optical signals, with significantly less energy loss than traditional designs, and because the conversion occurs so close to the silicon, the electrical path remains relatively short. Signal quality remains high, and all of the power that was previously squandered fighting electrical resistance and heat is now saved. That is now ready for practical computing.

Lasers are no longer placed within each individual transceiver. External laser source modules now exist in a regulated environment, feeding light into all of the optical engines at the same time. This architecture allows you to employ four times as few lasers as previous approaches. Fewer lasers result in less power, heat, and moving parts that can wear out or fail over time. NVIDIA claims that when compared to networks that employ traditional pluggable optics, you get five times the network power efficiency and five times the amount of uninterrupted AI application runtime. As a bonus, the average time between events increases by a factor of 10.

To give you an idea, the SN6810 can squeeze 128 800 Gb/s ports into a 2U liquid-cooled chassis while moving a whopping 102.4 terabits per second. The bigger SN6800 stacks four ASICs in a 5U system, delivering 409.6 terabits per second across 512 ports at 800 Gb/s, or over two thousand ports at 200 Gb/s. For those running large GPU clusters, an inbuilt fiber shuffle inside the larger chassis keeps connections very flat and straightforward, allowing you to expand without adding extra levels of switching that would simply increase latency.

To make this operate on a larger scale, assembly procedures had to be revised. Optical engines are soldered directly onto the module substrate using a manufacturing procedure that integrates well with normal production processes. A simple fiber connector allows you to attach fibers with great precision and success rate from the top surface. The various supply chain partners take on different aspects of the job: TSMC handles silicon photonics, SPIL handles chip-scale packaging and testing, Lumentum and TFC supply laser components, while Foxconn develops the entire switch system from the ground up. Before shipping, NVIDIA performs a last round of testing on the completed switches in its own facilities, and production is now fully underway.

These switches are merely one of Spectrum-X’s larger Ethernet platform. This platform uses adaptive routing, congestion control, and end-to-end telemetry to keep traffic flowing even when thousands of GPUs are attempting to communicate at the same time. With multiplane systems, each connection is separated across numerous distinct network planes, thus a single failure cannot bring an entire project to a standstill. Spectrum-XGS then extends the same network fabric over entire buildings or campuses, essentially combining multiple data centers into a single coordinated system. The technology claims to improve AI networking speed by 1.6 times over traditional Ethernet while remaining open and compatible with popular tools such as SONiC.