What connects ten thousand GPUs into one AI model?
It starts simply: one accelerator, one small matrix. It fits on one chip, so the problem is compute. Scale up to eight GPUs, then a full rack, then thousands of accelerators in a single cluster. But the model no longer fits on one chip, so the work splits across every GPU. Each GPU holds just a slice of the layers.
Now they must constantly exchange what they learn. Each GPU computes a gradient, one number to share. For GPUs, report zero point seven, zero point nine, zero point six, and zero point eight. The network averages them into a single value. Add them, divide by four, and you get zero point seven five. Then zero point seven five travels back to every GPU. That round trip is called all reduce. Training repeats this on every single step.
All that traffic must cross the network. The links fill up, and packets start to queue. Now the expensive GPUs sit waiting for data. Compute finishes, but the next step can't start.
AI traffic doesn't look like normal web browsing. One GPU can blast data to thousands at once. Thousands can hammer one GPU at the same time. That pileup has a name: incast. Traffic converges, and one link becomes a jam.
Normally, data climbs through the CPU and kernel. RDMA lets a network card read remote memory directly. The CPU steps aside, so latency drops. Put that on Ethernet, and you get Rose. But RDMA alone doesn't remove congestion or loss.
AI needs transport built for its own traffic. Metrosy sprays each packet across many paths. Lose a packet, and only that packet returns. It manages congestion from both ends of a flow.
At this scale, the network stops being plumbing. It becomes part of the computer itself. So remember, those GPUs still have to talk.
Likely speech-to-text slips, left as spoken: "For GPUs" is probably "Four GPUs", "Rose" is probably RoCE, and "Metrosy" is a name I can't identify with confidence.