Publication
The collective communication libraries are pivotal in optimizing the performance of distributed and parallel deep neural network (DNN) training. Most network optimizations are under the assumption of fixed collective communication algorithms and parameters, limiting the potential performance gains. AutoCCL proposes an automated framework that tunes collective communication configurations to significantly improve training throughput.
Guanbin Xu, Zhihao Le, Yinhe Chen, Zhiqi Lin, Zewen Jin, Youshan Miao, Cheng Li
The collective communication libraries are pivotal in optimizing the performance of distributed and parallel deep neural network (DNN) training. AutoCCL proposes an automated framework that tunes collective communication configurations to significantly improve training throughput.