Neural Network Pruning

Neural network pruning is a prominent technique to compress a neural network by reducing the number of parameters (or the number of neurons) which is often desirable for resource-limitted devices and/or real-time applications. In this project, we develop new pruning methods specifically targetting pruning at initialization and attempt to theoretically understand the training of sparse networks as well as evaluate the scalability of such pruning at initialization methods.

Understanding the Effects of Data Parallelism and Sparsity on Neural Network Training.
Namhoon Lee, Thalaiyasingam Ajanthan, Philip H. S. Torr, and Martin Jaggi.
International Conference on Learning Representations (ICLR), May 2021.
[pdf] [arxiv] [bib]

@article{lee_dataparallelism_iclr21,
  author = {Lee, Namhoon and Ajanthan, Thalaiyasingam and Torr, Philip HS and Jaggi, Martin},
  title = {Understanding the Effects of Data Parallelism and Sparsity on Neural Network Training},
  journal = {ICLR},
  year = {2021}
}

RANP: Resource Aware Neuron Pruning at Initialization for 3D CNNs.
Zhiwei Xu, Thalaiyasingam Ajanthan, Vibhav Vineet, and Richard Hartley.
International Conference on 3D Vision (3DV), November 2020. (oral, best student paper)
[pdf] [arxiv] [talk] [code] [bib]

@article{xu_ranp_3dv20,
  author = {Xu, Zhiwei, and Ajanthan, Thalaiyasingam, and Vineet, Vibhav, and Hartley, Richard},
  title = {RANP: Resource Aware Neuron Pruning at Initialization for 3D CNNs},
  journal = {3DV},
  year = {2020}
}

A Signal Propagation Perspective for Pruning Neural Networks at Initialization.
Namhoon Lee, Thalaiyasingam Ajanthan, Stephen Gould, and Philip H. S. Torr.
International Conference on Learning Representations (ICLR), April 2020. (spotlight)
[pdf] [arxiv] [talk] [code] [bib]

@article{lee_disnip_iclr20,
  author = {Lee, Namhoon and Ajanthan, Thalaiyasingam and Gould, Stephen and Torr, Philip HS},
  title = {A Signal Propagation Perspective for Pruning Neural Networks at Initialization},
  journal = {ICLR},
  year = {2020}
}

SNIP: Single-shot Network Pruning based on Connection Sensitivity.
Namhoon Lee, Thalaiyasingam Ajanthan, and Philip H. S. Torr.
International Conference on Learning Representations (ICLR), May 2019.
[pdf] [arxiv] [poster] [talk] [code] [bib]

@article{lee_snip_iclr19,
  author = {Lee, Namhoon and Ajanthan, Thalaiyasingam and Torr, Philip HS},
  title = {{SNIP:} Single-shot Network Pruning based on Connection Sensitivity},
  journal = {ICLR},
  year = {2019}
}