Scale-up networking has been the part of an AI cluster that network teams mostly didn’t own. Inside an NVL72 rack it’s NVLink and NVSwitch, designed and sold with the GPU. Ethernet started at the NIC and went outward. On October 7, ahead of next week’s OCP Global Summit in San Jose, Arista announced a set of rack designs that push Ethernet into that inner domain, under the name Etherlink SU-144.

The headline is 144 accelerators in a single-hop Ethernet scale-up domain inside one rack, and up to 1,024 across racks, with AMD, Arm, Broadcom, d-Matrix, Meta, Microsoft, and Qualcomm named as partners. What’s more useful is underneath: the protocol, the three physical layouts, and what’s still only a reference design.

What ESUN actually changes on the wire

Etherlink SU-144 is built on ESUN, Ethernet for Scale-Up Networking, an Open Compute Project workstream that started at the 2025 OCP summit with twelve founding members (AMD, Arista, Arm, Broadcom, Cisco, HPE Networking, Marvell, Meta, Microsoft, NVIDIA, OpenAI, Oracle). OCP says it had grown past 175 participating companies by the time the ESUN 1.0 specification shipped in March 2026. Meta and Microsoft led the spec.

The spec is specific about three things.

A smaller header. ESUN defines a 4-byte ESUN Header that replaces the IP/UDP stack, which OCP puts at 28 to 48 bytes. Scale-up traffic is dominated by small messages, like tensor-parallel all-reduce fragments and MoE expert-routing payloads, where per-packet overhead is a real fraction of the link.

To put a number on it, by my arithmetic: take a 256-byte payload and count the usual Ethernet framing costs on the wire (8 bytes preamble and start delimiter, 14 bytes Ethernet header, 4 bytes FCS, 12 bytes inter-frame gap, 38 bytes in all). With IPv4 plus UDP on top, the payload is about 79.5% of the bits on the wire. With IPv6 plus UDP it’s about 74.9%. With the 4-byte ESUN header it’s about 85.9%. That ignores whatever transport header sits above, so treat it as an upper bound on the gain, not a measured goodput number. Still, six to eleven points of link efficiency on small messages is the kind of thing you’d otherwise pay for with more lanes.

Lossless delivery at the link layer. The ESUN base spec requires Priority Flow Control on switches and adds two mechanisms defined by the Ultra Ethernet Consortium. Link Level Retry (LLR) retransmits a frame between the two peers on a link when forward error correction couldn’t fix it, instead of waiting for the endpoints to notice a missing packet and retransmit end to end. Credit Based Flow Control (CBFC) lets a sender transmit only when the receiver has advertised buffer space, which the spec says allows more memory-efficient switch designs than PFC alone. Both are configurable per port.

At 200G-per-lane SerDes, FEC alone leaves a tail of uncorrectable frames, and in a scale-up collective one end-to-end retransmit stalls every rank. Fixing it on the hop where it happened costs far less time than an end-to-end timeout and retransmit. The spec is explicit that LLR only covers bit-error drops; congestion drops and anything else still need end-to-end recovery.

Congestion control for multi-hop. The ESUN header carries ECN, class of service, TTL, and a flow label, so a multi-hop scale-up topology can signal both transient and persistent congestion. That’s what a scale-up domain needs once it stops being a single switch hop, as in the cross-rack design.

ESUN is the network-side spec. The endpoint side, how an accelerator turns loads, stores and collectives into packets, is a separate OCP workstream called SUE-T, seeded by Broadcom’s Scale-Up Ethernet 1.0 contribution. Arista’s announcement is about switches, racks and diagnostics. It doesn’t ship an endpoint.

Three ways to build the rack

Arista’s engineering blog, written by Arihant Jain and Vijay Vusirikala, and the analyst briefing reported by SiliconANGLE describe three physical layouts.

  • Orthogonal chassis. Accelerator blades in front, full-height switch cards in back, connected directly with no backplane and no cables. This is where the 144-XPU single-hop number lives, in a 100 to 400 kW direct liquid cooling envelope. It’s the design aimed at training.
  • Cabled backplane. Switches in the middle of the rack, connected to servers with copper cable cartridges. Less dense, more serviceable, and it lets an operator split switch bandwidth between intra-rack and inter-rack links.
  • Cross-rack. A dedicated switch rack ties separate accelerator racks together. This is the route to 1,024 XPUs. Vusirikala called it the most flexible and fastest to market, since it skips long co-design with the accelerator vendor, at a cost in interconnect and power. His line on reach was blunt: “Copper runs out of juice very quickly,” so anything beyond two racks needs optics.

That last point is the trade I’d want to model before believing 1,024. A cross-rack scale-up domain on optics adds transceiver power per lane and optical latency on every hop, and it brings optical failure modes into the part of the cluster least tolerant of them. None of that is quantified in the announcement.

The scale-out half and the numbers you can check

The scale-out side is more conventional and easier to verify. Arista offers liquid-cooled rack reference designs with 8, 12, or 16 of its 7060EX7 switches, each 102.4 Tbps. Sixteen of those is 1,638 Tbps, which is the “1.6 Pb/s per rack” figure in the release. Arista says next-generation optics, its XPO pluggables or open co-packaged and near-packaged optics, take that to 6.5 Pb/s per rack. Hardev Singh, who runs Arista’s cloud and AI group, told Network World that “a Tomahawk 7, 200G-based generation” could support that 6.5 figure in a single rack drawing more than 100 kW. The “nearly 50% reduction in overall data center footprint” that follows is Arista’s claim, not something I can check from the outside.

The racks include patch panels, cooling manifolds, drip trays with active leak detection, and battery-backed power shelves. The fabric supports UEC features, Multipath Reliable Connection, and multi-plane routing, and it’s meant to sit beside whatever scale-up protocol the compute uses: ESUN, NVLink, NVLink Fusion, or PCIe.

Diagnostics are the real product

The part that will decide whether these racks work in production is Netdi, Arista’s Network Diagnostics Infrastructure. Per the blog, it does hardware-level validation, signal integrity analysis, secure boot attestation, and single event upset resiliency, with telemetry covering switches, optics, power shelves, and the liquid cooling plant. It’s NOS-agnostic, so it runs under EOS, SONiC, FBOSS, or another network OS.

Jain’s operational argument in the briefing: a 16-switch rack can involve about 2,000 fibers, and assembling that on site invites miswiring, bend-radius violations, and dead optics, so racks should be integrated and tested upstream, then rolled in.

From the cluster side, this is what I care about most, because a scale-up domain fails as a unit. If one link in a 144-XPU group flaps, the job sharded across it stalls or restarts, and the scheduler has to know which nodes to cordon. Diagnostic telemetry that can isolate a bad optic or a marginal SerDes lane before the job dies is worth more than a few percent of bandwidth. Whether Netdi exposes that in a form a cluster health system can consume, through gNMI, OpenTelemetry, or something a Kubernetes node-problem detector can read, is the question I’d ask at the booth.

What this is not

A few limits are stated plainly in the coverage, and they matter:

  • It isn’t a SKU. Singh told Network World that customers won’t find a rack on Arista’s price list. Arista sells the switches, software, and validation, and the integrated rack comes from partners and system integrators such as Foxconn and Quanta.
  • It doesn’t touch NVIDIA scale-up. Vusirikala told SiliconANGLE there is “no near-term opportunity” inside NVIDIA’s scale-up domain because it is proprietary. The market here is AMD Helios, Microsoft Maia, Meta MTIA, Qualcomm, d-Matrix, and other non-NVIDIA accelerators.
  • There’s no latency number. No port-to-port figure and no collective benchmark, and for scale-up that’s the number that matters most.
  • There’s no ship date for the scale-up racks themselves.

Where I land

The useful signal isn’t that Arista wants a piece of scale-up. It’s that ESUN 1.0 is specific enough, with a 4-byte header, link-level retry, credit-based flow control, and a multi-hop congestion header, that a switch vendor can build three different physical racks against it and partners can build accelerators to the other side. That’s roughly how Ethernet caught up with InfiniBand in scale-out, a debate SiliconANGLE’s Zeus Kerravala calls “largely settled”: not on peak numbers, but on a common spec that many vendors could ship to.

If you run clusters on non-NVIDIA accelerators, or expect to in a 2027 build, the questions I’d take to OCP next week are concrete. What’s the measured all-reduce latency across 144 XPUs in the orthogonal chassis, versus across the cross-rack design on optics? What’s the LLR retry rate at 200G per lane in a real liquid-cooled rack? And what does Netdi emit when a link starts to degrade, and how fast? Arista will have a fully assembled liquid-cooled rack at booth E61, and Andy Bechtolsheim is speaking at 1 p.m. on October 13. I’d go for the rack and the telemetry, not the keynote slide.

Create-only. Specs from Arista’s October 7, 2026 press release and engineering blog, the OCP ESUN 1.0 specification, Network World, and SiliconANGLE.

Sources: Arista press release, Oct 7 · Arista engineering blog · OCP ESUN 1.0 specification · ESUN base spec · SUE-T · Network World · SiliconANGLE.

Related: Helios UALoE · Cornelis Delos · openTPU.

Anpoo Sivanadi, Staff Software Engineer