HKBU Computer Scientist wins Best Paper Award at IEEE INFOCOM 2021

Prof. Xiaowen Chu, HKBU, along with Dr Shaohuai Shi, PhD Graduate of the HKBU Department of Computer Science and Prof. Bo Li, HKUST, received the Best Paper Award for their co-authored paper “Exploiting Simultaneous Communications to Accelerate Data Parallel Distributed Deep Learning” at the IEEE International Conference on Computer Communications (IEEE INFOCOM 2021).

The award-winning paper proposes a novel algorithm to reduce the training time of large AI models when running on GPU clusters. Through theoretical analysis and experiments, it demonstrates that simultaneous All-Reduce communications can effectively improve the communication efficiency of small tensors. To exploit both tensor fusion and simultaneous communications, the team formulated an optimis ation problem and developed an efficient solution named ASC-WFBP. The team also ran real-world experiments on an 8-node GPU cluster with 32 GPUs and 10Gbps Ethernet. Tested on four modern AI models, their experimental findings indicated that ASC-WFBP could achieve speedups of around 1.09x - 2.48x over the baseline without tensor fusion, and 1.15x - 1.35x over the state-of-the-art tensor fusion solution.

The 4-day Conference on Computer Communications (INFOCOM) is a top-ranked conference on networking in the research community. It is for researchers to present and exchange significant and innovative contributions and ideas in the field of networking and closely related areas. Out of 1,266 submissions, only one-fifth were accepted after a rigorous double-blind review process, and three papers were selected to receive the Best Paper Award.