← Research

Learning Dynamics

Optimization behavior, stability, and trainability in deep networks.

2026

NeurIPS

NuMuon: Nuclear-Norm-Constrained Muon for Compressible LLM Training

H. Mohaghegh Dolatabadi, T. Ajanthan, S. Ramasinghe, C. P. Hewa Koneputugodage, S. Siriwardhana, V. Shevchenko, K. Pajak, J. Snewin, G. Avraham, A. Long

PDFarXiv
BibTeX
@inproceedings{dolatabadi_numuon_arxiv26,
  title={{NuMuon: Nuclear-Norm-Constrained Muon for Compressible LLM Training}},
  author={Mohaghegh Dolatabadi, Hadi and Ajanthan, Thalaiyasingam and Ramasinghe, Sameera and P Hewa Koneputugodage, Chamin and Siriwardhana, Shamane and Shevchenko, Violetta and Pajak, Karol and Snewin, James and Avraham, Gil and Long, Alexander},
  year={2026},
  booktitle={NeurIPS}
}
ICLR

Taming Curvature: Architecture Warm-up for Stable Transformer Training

S. Ramasinghe, T. Ajanthan, H. Mohaghegh Dolatabadi, C. P. Hewa Koneputugodage, G. Avraham, V. Shevchenko, Y. Zuo, K. Pajak, A. Long

PDFarXiv
BibTeX
@inproceedings{ramasinghe_archwp_iclr26,
  title={{Taming Curvature: Architecture Warm-up for Stable Transformer Training}},
  author={Ramasinghe, Sameera and Ajanthan, Thalaiyasingam and Mohaghegh Dolatabadi, Hadi and P Hewa Koneputugodage, Chamin and Avraham, Gil and Shevchenko, Violetta and Zuo, Yan and Pajak, Karol and Long, Alexander},
  year={2026},
  booktitle={ICLR}
}

2024

2023

2021

2019