Abstract

Neural network pruning is a prominent technique to compress a neural network by reducing the number of parameters (or the number of neurons) which is often desirable for resource-limitted devices and/or real-time applications. In this project, we develop new pruning methods specifically targetting pruning at initialization and attempt to theoretically understand the training of sparse networks as well as evaluate the scalability of such pruning at initialization methods.

Publications

Understanding the Effects of Data Parallelism and Sparsity on Neural Network Training.
Namhoon Lee, Thalaiyasingam Ajanthan, Philip H. S. Torr, and Martin Jaggi.
International Conference on Learning Representations (ICLR), May 2021.
[pdf] [arxiv] [bib]

RANP: Resource Aware Neuron Pruning at Initialization for 3D CNNs.
Zhiwei Xu, Thalaiyasingam Ajanthan, Vibhav Vineet, and Richard Hartley.
International Conference on 3D Vision (3DV), November 2020. (oral, best student paper)
[pdf] [arxiv] [talk] [code] [bib]

A Signal Propagation Perspective for Pruning Neural Networks at Initialization.
Namhoon Lee, Thalaiyasingam Ajanthan, Stephen Gould, and Philip H. S. Torr.
International Conference on Learning Representations (ICLR), April 2020. (spotlight)
[pdf] [arxiv] [talk] [code] [bib]

SNIP: Single-shot Network Pruning based on Connection Sensitivity.
Namhoon Lee, Thalaiyasingam Ajanthan, and Philip H. S. Torr.
International Conference on Learning Representations (ICLR), May 2019.
[pdf] [arxiv] [poster] [talk] [code] [bib]