Scale-Up vs Scale-Out 2026: NVLink, UALink, Ultra Ethernet

Q: What is the difference between a scale-up and a scale-out interconnect in an AI cluster?

A scale-up interconnect joins a small group of accelerators so tightly that they behave as one machine: another GPU's memory is a load/store target, latency is measured in hundreds of nanoseconds to a few microseconds, and the fabric is lossless. NVLink, UALink and the Ethernet-based scale-up profiles (Broadcom SUE, AMD's UALink over Ethernet, OCP ESUN, UEC 1.1 Unified Local) are scale-up fabrics, and in every shipping product in 2026 the domain is 72 GPUs in one liquid-cooled rack. A scale-out interconnect joins those machines into a cluster with a network: Ultra Ethernet or InfiniBand carried over NICs, tolerant of loss and reordering, scaling to millions of endpoints. Tensor-parallel and expert-parallel collectives need the scale-up fabric; pipeline, data-parallel and disaggregated prefill/decode traffic belongs on scale-out.

Q: How do I decide how large my scale-up domain needs to be?

Take the largest model you will run during the hardware's life at the precision you will actually serve, add the KV cache at the batch size that makes the economics work, and divide by the HBM per accelerator. That quotient is the minimum tensor-parallel group and it must fit inside one scale-up domain with headroom for the next model generation. For a mixture-of-experts model the expert-parallel group must also fit, because an all-to-all that crosses the domain boundary becomes a network operation on every layer. For most enterprise inference estates in 2026 the answer is 8 GPUs, which any vendor's node provides; the 72-GPU, 260 TB/s racks from NVIDIA and AMD exist for frontier training and for serving trillion-parameter models as one machine.

Q: What did Ultra Ethernet 1.1 change on 12 October 2026?

UE Specification 1.1 extended Ultra Ethernet from scale-out into scale-up. It adds a Unified Local transport profile for accelerator-to-accelerator write, read and atomic operations over short-range links, a Unified Forwarding Header that overlays the Ethernet header and saves 20 bytes per packet versus a traditional IP packet, reduced inter-packet gap for small-message efficiency, in-network collectives in which switches reduce data on arrival and broadcast the result so an endpoint sends half as much for an all-reduce, Congestion Signaling (CSIG) tags of 4 or 8 bytes updated by switches, and Programmable Congestion Management so congestion control and load balancing run as software on any PCM-capable NIC. The consortium's technical chair has said the features need some hardware support and expects vendors to enable them on existing hardware through firmware; compliance remains self-attested.

Q: Is UALink shipping in 2026, and what is UALink over Ethernet?

Native UALink switch silicon was not shipping when the UALink Consortium published its 2.0 specification set on 7 April 2026; the 2026 member news is prototypes, retimers, IP blocks, security modules and test equipment. UALink 1.0 defines 200 GT/s per lane, four-lane stations carrying 800 Gb/s in each direction, sub-microsecond round trips, cable reach under four metres and a theoretical maximum of 1,024 accelerators in a pod. UALink over Ethernet, which AMD calls the UALoE fabric in its Helios rack launched on 23 July 2026, carries the UALink protocol over merchant Ethernet switch silicon such as Broadcom Tomahawk 6, which is how AMD ships a 72-GPU, 260 TB/s scale-up domain in 2026 without waiting for native UALink switches. The trade is that the protocol layering is first-generation and defined alongside three other Ethernet scale-up efforts.

Q: How does NVIDIA NVLink 6 on Vera Rubin NVL72 compare with AMD Helios?

Both connect 72 GPUs in one scale-up domain at 260 TB/s of aggregate bandwidth, which is 3.6 TB/s bidirectional per GPU. NVIDIA's Rubin platform documentation gives 288 GB of HBM4 per GPU at up to 22 TB/s, NVLink 6 switch trays at 28.8 TB/s each with FP8 in-network compute, and ConnectX-9 NICs at 1.6 Tb/s per GPU for scale-out. AMD's Helios launch states 31 TB of HBM4 across the rack, 1.7 PB/s of aggregate HBM bandwidth, 43 TB/s of scale-out bandwidth and a design claim of up to 15 percent more AI compute, 50 percent more HBM capacity and 50 percent more scale-out bandwidth than an NVL72 rack; the networking figures behind that comparison are AMD simulations. The decisive difference is the fabric: NVLink is a single-vendor switch with hardware memory semantics and a tuned software stack, while Helios uses UALink over merchant Ethernet switches available from multiple OEMs.

Q: Should I buy co-packaged optics switches in 2026?

For the scale-out spine, when your port count and power budget justify it, yes. NVIDIA lists Spectrum-X Ethernet Photonics at up to 409.6 Tb/s as available in the second half of 2026 with CoreWeave, Lambda, Meta, Microsoft and Oracle Cloud Infrastructure as first adopters, and Quantum-X InfiniBand Photonics with 144 ports at 800 Gb/s; the vendor claims 5x better power efficiency than pluggable transceivers and 5x sustained application runtime from fewer link flaps. Those are real savings at million-GPU scale because transceiver power and transceiver failures are a material part of the bill. For the scale-up domain, no: every proven domain in 2026 is copper inside one rack, and the NVLink roadmap to 1,152-accelerator domains and the UALink 1,024 figure both depend on optics in the scale-up path that are not in volume production. Plan serviceability for liquid-cooled optics before you deploy any CPO.

Q: What does "exit cost" mean when choosing an interconnect, and how do I reduce it?

Exit cost is what it takes to run the next generation of accelerators, or another vendor's accelerators, on the fabric you own. With NVLink the domain is a single-vendor machine: switch, GPUs, NCCL and the serving stack move together, and NVLink Fusion opens only the CPU and custom-XPU ends of the fabric, not the switch. With Ethernet-based scale-up the switches and cabling are standard, so the exit cost is on the accelerator side and in the convergence risk between SUE, UALoE, ESUN 1.0 and UEC 1.1 Unified Local. To reduce it, name the specification profile and version the switch firmware must implement in the contract, require an interoperability test report rather than a compliance claim, benchmark the collective at your domain size before acceptance, keep every scale-out link on open Ultra Ethernet, and never bridge two scale-up fabrics inside one domain.

Q: What are the most common mistakes when specifying an AI cluster interconnect?

Ranking fabrics by bandwidth when two 260 TB/s racks differ in protocol, switch supply, software and exit cost; buying a 72-GPU domain for workloads whose largest model fits in 8 GPUs; filling the scale-up fabric with disaggregated prefill/decode, data-parallel and orchestration traffic that belongs on Ethernet; specifying native UALink as if switches were shipping in 2026; mixing two scale-up protocols inside one domain; treating "open" as a synonym for "interoperable" when four Ethernet scale-up definitions coexist; buying co-packaged optics for the scale-up domain rather than the scale-out spine; and placing a mixture-of-experts model's expert-parallel group across two domains so that every all-to-all becomes a network operation. Each of these is avoided by the same rule: size the domain to the collective first, then choose the fabric by exit cost.

Satyam Kumar

返回博客

architecture

Scale-Up vs Scale-Out in 2026: The Interconnect Sets the Size of Your Machine, So Size the Domain to the Collective and Buy the Fabric by Its Exit Cost, Not Its Bandwidth Number

By Satyam KumarOctober 22, 202620 min read

scale-up vs scale-out nvlink 6 ualink 2.0 ultra ethernet 1.1 ualink over ethernet amd helios vera rubin nvl72 co-packaged optics spectrum-x photonics broadcom tomahawk ultra scale-up ethernet ocp esun ai cluster interconnect gpu interconnect topology nvlink fusion rack-scale ai

Scale-Up vs Scale-Out in 2026: The Interconnect Sets the Size of Your Machine, So Size the Domain to the Collective and Buy the Fabric by Its Exit Cost, Not Its Bandwidth Number

Frequently Asked Questions

分享这篇文章

Twitter LinkedIn WhatsApp

Satyam Kumar

Founder & AI Architect, AppScale LLP

人工智能和云架构师。帮助团队构建可扩展到数百万的系统。

LinkedIn GitHub

Scale-Up vs Scale-Out in 2026: The Interconnect Sets the Size of Your Machine, So Size the Domain to the Collective and Buy the Fabric by Its Exit Cost, Not Its Bandwidth Number

Frequently Asked Questions

分享这篇文章

Comments

Leave a comment