Collaborative Channel Pruning for Deep Networks

Hanyu Peng, Jiaxiang Wu, Shifeng Chen, Junzhou Huang
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:5113-5122, 2019.

Abstract

Deep networks have achieved impressive performance in various domains, but their applications are largely limited by the prohibitive computational overhead. In this paper, we propose a novel algorithm, namely collaborative channel pruning (CCP), to reduce the computational overhead with negligible performance degradation. The joint impact of pruned/preserved channels on the loss function is quantitatively analyzed, and such interchannel dependency is exploited to determine which channels to be pruned. The channel selection problem is then reformulated as a constrained 0-1 quadratic optimization problem, and the Hessian matrix, which is essential in constructing the above optimization, can be efficiently approximated. Empirical evaluation on two benchmark data sets indicates that our proposed CCP algorithm achieves higher classification accuracy with similar computational complexity than other stateof-the-art channel pruning algorithms

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-peng19c, title = {Collaborative Channel Pruning for Deep Networks}, author = {Peng, Hanyu and Wu, Jiaxiang and Chen, Shifeng and Huang, Junzhou}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {5113--5122}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/peng19c/peng19c.pdf}, url = {https://proceedings.mlr.press/v97/peng19c.html}, abstract = {Deep networks have achieved impressive performance in various domains, but their applications are largely limited by the prohibitive computational overhead. In this paper, we propose a novel algorithm, namely collaborative channel pruning (CCP), to reduce the computational overhead with negligible performance degradation. The joint impact of pruned/preserved channels on the loss function is quantitatively analyzed, and such interchannel dependency is exploited to determine which channels to be pruned. The channel selection problem is then reformulated as a constrained 0-1 quadratic optimization problem, and the Hessian matrix, which is essential in constructing the above optimization, can be efficiently approximated. Empirical evaluation on two benchmark data sets indicates that our proposed CCP algorithm achieves higher classification accuracy with similar computational complexity than other stateof-the-art channel pruning algorithms} }
Endnote
%0 Conference Paper %T Collaborative Channel Pruning for Deep Networks %A Hanyu Peng %A Jiaxiang Wu %A Shifeng Chen %A Junzhou Huang %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-peng19c %I PMLR %P 5113--5122 %U https://proceedings.mlr.press/v97/peng19c.html %V 97 %X Deep networks have achieved impressive performance in various domains, but their applications are largely limited by the prohibitive computational overhead. In this paper, we propose a novel algorithm, namely collaborative channel pruning (CCP), to reduce the computational overhead with negligible performance degradation. The joint impact of pruned/preserved channels on the loss function is quantitatively analyzed, and such interchannel dependency is exploited to determine which channels to be pruned. The channel selection problem is then reformulated as a constrained 0-1 quadratic optimization problem, and the Hessian matrix, which is essential in constructing the above optimization, can be efficiently approximated. Empirical evaluation on two benchmark data sets indicates that our proposed CCP algorithm achieves higher classification accuracy with similar computational complexity than other stateof-the-art channel pruning algorithms
APA
Peng, H., Wu, J., Chen, S. & Huang, J.. (2019). Collaborative Channel Pruning for Deep Networks. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:5113-5122 Available from https://proceedings.mlr.press/v97/peng19c.html.

Related Material