Start with the smallest case: one graphics processor (GPU), working on one small table of numbers. It all fits on the chip, so the only limit is how fast the chip can calculate.
Now scale up to eight GPUs, then a full rack of them, then thousands in one cluster. A large AI model no longer fits on any single chip. So the model is cut up, and each GPU holds just a slice of its layers. Each chip does its share of the work, but the chips can't work in isolation, because the model only makes sense as a whole.
After each step of training, every GPU works out a gradient: one number saying how the model should adjust itself. Each chip has only seen its own slice, so the chips must pool their results.
Take four GPUs reporting 0.7, 0.9, 0.6 and 0.8. The network adds them up and divides by four, giving 0.75. That single value then goes back out to every chip, so they all adjust the same way. This round trip is called all-reduce ("everyone sends in a number, everyone gets the average back"). Training repeats it on every single step, thousands upon thousands of times.
All of that traffic has to cross the network. When the links fill up, data queues behind other data. The chips are very expensive and finish their calculations quickly, but then they sit idle, waiting for the averaged result. They can't begin the next step without it. That is why the next step can't start: the answer the next step depends on is stuck in traffic.
Ordinary web browsing is spread out and unpredictable. AI training is the opposite. One GPU may send data to thousands of others at once, or thousands may send to one GPU at the same moment. When many senders converge on one receiver, that receiver's link turns into a traffic jam. This pile-up is called incast.
A network card (NIC) already uses direct memory access (DMA) to move data into its own machine's memory without bothering the processor. RDMA (Remote Direct Memory Access) extends that across the network: the sending card writes straight into the memory of a different machine, and that machine's processor and operating system stay out of the loop. Delay drops.
When RDMA runs over ordinary Ethernet cabling, the result is called RoCE (RDMA over Converged Ethernet). The transcript renders this as "Rose".
Fast delivery doesn't remove congestion or lost data, though. AI training needs a transport method designed for its own traffic pattern. The one described works in three ways:
The transcript garbles the name of this method ("Metrosy"), so I can't name it with confidence.
At this scale, the network is no longer just plumbing that carries data between computers. It is part of the computer itself. If the chips can't talk to each other quickly, the whole machine slows down, however fast each chip is.