📘 Resource Overview & Documentation Details

You are viewing the technical directory listing for In-Network Aggregation: Faster Distributed ML Training. This engineering resource is curated under our Machine Learning division to support automation specialists, control systems engineers, and manufacturing technicians looking for verified reference materials.

🔍 Contextual Abstract & Technical Scope

Share Summary:LinkedInEmail

In distributed machine learning, adding workers and faster accelerators does not automatically make training faster. Each worker must share its model updates with others, creating substantial network traffic that can become a bottleneck. In-network aggregation offers another approach: instead of sending every update to worker hosts for combination, programmable switches perform part of the aggregation as data moves through the network. SwitchML demonstrates how this design can support distributed training while working within the processing and storage limits of network hardware.

Why distributed training puts pressure on the network

In data-parallel training, each worker maintains a local copy of a model and processes data independently. After a training step, workers exchange their updates and aggregate them so that the model copies remain synchronized. This exchange is repeated throughout training.

Model updates can amount to hundreds of megabytes per iteration. Moving and combining them across multiple workers creates intensive all-to-all communication. As accelerators make computation faster, the time spent exchanging updates can take a larger share of the training cycle. The result is that improving compute capacity alone may not address the overall training bottleneck.

Conventional approaches commonly perform aggregation on worker hosts. SwitchML changes where some of that work happens: a programmable switch combines incoming updates, reducing the need for hosts to receive and process every contribution individually.

How SwitchML uses in-network aggregation

SwitchML is a system for aggregating model updates inside the network. Its design treats the switch as a participant in distributed training, rather than only as a device that forwards packets. The switch aggregates update values and returns the combined result to workers.

That approach has to account for constraints that differ from those of a server. A switch has limited storage and computation, and the data plane does not provide floating-point operations. Packet loss also has to be handled without corrupting an aggregation or applying a worker’s contribution more than once.

Streaming aggregation within a limited switch pool

Keeping an entire model’s updates in switch memory is not practical for this design. SwitchML instead uses a pool-based streaming approach. Workers divide update vectors into chunks, and the switch processes those chunks through a limited pool of aggregation slots. This makes it possible to handle large updates without requiring the switch to store the complete model.

The system divides responsibilities between workers and the switch. Worker software chunks vectors, quantizes values, and detects and recovers from packet drops. The switch adds integer values, selects slots, and tracks whether the contributions needed for a slot have arrived. The switch program was implemented in P4 for Barefoot Tofino hardware, with an end-host C++ library providing an all-reduce interface.

Quantization for switch-compatible arithmetic

Because the switch cannot perform floating-point aggregation, SwitchML converts floating-point updates to 32-bit fixed-point values using a scaling factor. The design describes two conversion paths: converting between 16-bit floating point and 32-bit fixed point in the switch, or performing 32-bit floating-point-to-fixed-point conversion at workers using AVX instructions.

Quantization introduces a practical question: whether representing updates this way harms model training. The reported results indicate that, across a large range of scaling factors, training achieved similar accuracy in a similar number of iterations to training with unquantized network communication. The scaling factor is therefore part of the numerical design, not simply a switch setting.

Handling packet loss and duplicate updates

Reliable aggregation requires more than adding values quickly. Packets can be lost while workers send updates or while the switch distributes aggregated results. Workers use timers to detect losses and retransmit packets, while the protocol prevents retransmissions from causing a contribution to be counted twice.

SwitchML tracks each worker’s contribution to a slot with a bitmap, allowing duplicate updates to be ignored. It also keeps a shadow copy of the previous result for a slot so a dropped result packet can be retransmitted. These mechanisms help preserve correct aggregation in the presence of packet loss. In the reported evaluation, reasonable packet-loss rates did not affect performance, and SwitchML showed lower performance inflation than TCP under loss.

What the evaluation reports

The evaluation used a testbed of 16 servers and a 64-port, 100-Gbps Barefoot Tofino switch. The servers included systems with P100 GPUs, and the tests covered nine standard CNN benchmarks, including ImageNet training except for the synthetic-data AlexNet case. SwitchML was compared with TensorFlow using NVIDIA Collective Communications Library (NCCL), including direct GPU memory access.

Across the evaluated workloads, SwitchML reported speedups ranging from 20% to 300% over that comparison. The evaluation also reported that performance did not depend on the number of workers. These are results from the described testbed and workloads; they should be understood in that context rather than as a guarantee for every training environment.

The system was integrated with TensorFlow through Horovod and with PyTorch/Caffe2 through Gloo. This reflects an effort to connect network-side aggregation with familiar machine-learning software interfaces, while keeping responsibilities split between the host and programmable switch.

Practical implications and remaining challenges

For teams operating distributed training infrastructure, SwitchML illustrates a broader systems principle: network devices can contribute to computation when a workload has a suitable, repetitive operation. Aggregating updates is a candidate because it involves combining values from multiple workers, but implementing it in a data plane requires careful choices about arithmetic, memory use, packet formats, and failure recovery.

The work also identifies areas for further development. These include coordinating multiple switches across racks, supporting multiple jobs or tenants by partitioning aggregation slots, exploring numeric representations that do not require a selected scaling factor, and carrying more data per packet. The authors note that full-MTU packets could improve performance by about 31% in their design, while the reported implementation reached line rate with small packets.

Conclusion

SwitchML shows how in-network aggregation can address the communication demands of distributed machine learning by moving update combination into a programmable switch. Its streaming design, fixed-point arithmetic, and loss-recovery protocol work together to fit aggregation into constrained data-plane hardware. The reported results suggest that this approach can accelerate training for the evaluated workloads, while its practical adoption depends on workload characteristics, network architecture, and the engineering requirements of integrating programmable switches with host-side training systems.

Access Official Technical PDF