Signal recovery from Pooling Representations

Joan Bruna Estrach, Arthur Szlam, Yann LeCun
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(2):307-315, 2014.

Abstract

Pooling operators construct non-linear representations by cascading a redundant linear transform, followed by a point-wise nonlinearity and a local aggregation, typically implemented with a \ell_p norm. Their efficiency in recognition architectures is based on their ability to locally contract the input space, but also on their capacity to retain as much stable information as possible. We address this latter question by computing the upper and lower Lipschitz bounds of \ell_p pooling operators for p=1, 2, ∞as well as their half-rectified equivalents, which give sufficient conditions for the design of invertible pooling layers. Numerical experiments on MNIST and image patches confirm that pooling layers can be inverted with phase recovery algorithms. Moreover, the regularity of the inverse pooling, controlled by the lower Lipschitz constant, is empirically verified with a nearest neighbor regression.

Cite this Paper

BibTeX
@InProceedings{pmlr-v32-estrach14, title = {Signal recovery from Pooling Representations}, author = {Estrach, Joan Bruna and Szlam, Arthur and LeCun, Yann}, booktitle = {Proceedings of the 31st International Conference on Machine Learning}, pages = {307--315}, year = {2014}, editor = {Xing, Eric P. and Jebara, Tony}, volume = {32}, number = {2}, series = {Proceedings of Machine Learning Research}, address = {Bejing, China}, month = {22--24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v32/estrach14.pdf}, url = {https://proceedings.mlr.press/v32/estrach14.html}, abstract = {Pooling operators construct non-linear representations by cascading a redundant linear transform, followed by a point-wise nonlinearity and a local aggregation, typically implemented with a \ell_p norm. Their efficiency in recognition architectures is based on their ability to locally contract the input space, but also on their capacity to retain as much stable information as possible. We address this latter question by computing the upper and lower Lipschitz bounds of \ell_p pooling operators for p=1, 2, ∞as well as their half-rectified equivalents, which give sufficient conditions for the design of invertible pooling layers. Numerical experiments on MNIST and image patches confirm that pooling layers can be inverted with phase recovery algorithms. Moreover, the regularity of the inverse pooling, controlled by the lower Lipschitz constant, is empirically verified with a nearest neighbor regression.} }
Endnote
%0 Conference Paper %T Signal recovery from Pooling Representations %A Joan Bruna Estrach %A Arthur Szlam %A Yann LeCun %B Proceedings of the 31st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2014 %E Eric P. Xing %E Tony Jebara %F pmlr-v32-estrach14 %I PMLR %P 307--315 %U https://proceedings.mlr.press/v32/estrach14.html %V 32 %N 2 %X Pooling operators construct non-linear representations by cascading a redundant linear transform, followed by a point-wise nonlinearity and a local aggregation, typically implemented with a \ell_p norm. Their efficiency in recognition architectures is based on their ability to locally contract the input space, but also on their capacity to retain as much stable information as possible. We address this latter question by computing the upper and lower Lipschitz bounds of \ell_p pooling operators for p=1, 2, ∞as well as their half-rectified equivalents, which give sufficient conditions for the design of invertible pooling layers. Numerical experiments on MNIST and image patches confirm that pooling layers can be inverted with phase recovery algorithms. Moreover, the regularity of the inverse pooling, controlled by the lower Lipschitz constant, is empirically verified with a nearest neighbor regression.
RIS
TY - CPAPER TI - Signal recovery from Pooling Representations AU - Joan Bruna Estrach AU - Arthur Szlam AU - Yann LeCun BT - Proceedings of the 31st International Conference on Machine Learning DA - 2014/06/18 ED - Eric P. Xing ED - Tony Jebara ID - pmlr-v32-estrach14 PB - PMLR DP - Proceedings of Machine Learning Research VL - 32 IS - 2 SP - 307 EP - 315 L1 - http://proceedings.mlr.press/v32/estrach14.pdf UR - https://proceedings.mlr.press/v32/estrach14.html AB - Pooling operators construct non-linear representations by cascading a redundant linear transform, followed by a point-wise nonlinearity and a local aggregation, typically implemented with a \ell_p norm. Their efficiency in recognition architectures is based on their ability to locally contract the input space, but also on their capacity to retain as much stable information as possible. We address this latter question by computing the upper and lower Lipschitz bounds of \ell_p pooling operators for p=1, 2, ∞as well as their half-rectified equivalents, which give sufficient conditions for the design of invertible pooling layers. Numerical experiments on MNIST and image patches confirm that pooling layers can be inverted with phase recovery algorithms. Moreover, the regularity of the inverse pooling, controlled by the lower Lipschitz constant, is empirically verified with a nearest neighbor regression. ER -
APA
Estrach, J.B., Szlam, A. & LeCun, Y.. (2014). Signal recovery from Pooling Representations. Proceedings of the 31st International Conference on Machine Learning, in Proceedings of Machine Learning Research 32(2):307-315 Available from https://proceedings.mlr.press/v32/estrach14.html.

Related Material