← Research

Context Parallelism

Distributing long-context computation across devices.

2025

NeurIPS

Mixtures of Subspaces for Bandwidth Efficient Context Parallel Training

S. Ramasinghe, T. Ajanthan, H. Mohaghegh Dolatabadi, G. Avraham, V. Shevchenko, Y. Zuo, C. P. Hewa Koneputugodage, A. Long

PDFarXiv
BibTeX
@inproceedings{ramasinghe_mixtures_neurips25,
  title={{Mixtures of Subspaces for Bandwidth Efficient Context Parallel Training}},
  author={Ramasinghe, Sameera and Ajanthan, Thalaiyasingam and Mohaghegh Dolatabadi, Hadi and Avraham, Gil and Shevchenko, Violetta and Zuo, Yan and P Hewa Koneputugodage, Chamin and Long, Alexander},
  year={2025},
  booktitle={NeurIPS}
}