Locally orderless networks

Jon Sporring, Peidi Xu, Jiahao Lu, Francois Bernard Lauze, Sune Darkner
Proceedings of the 6th Northern Lights Deep Learning Conference (NLDL), PMLR 265:239-244, 2025.

Abstract

We present Locally Orderless Networks (LON) and the theoretical foundation that links them to Convolutional Neural Networks (CNN), Scale-space histograms, and measurement theory. The key elements are a regular sampling of the bias and the derivative of the activation function. We compare LON, CNN, and Scale-space histograms on prototypical single-layer networks. We show how LON and CNN can emulate each other and how LON expands the set of functions computable to non-linear functions such as squaring. We demonstrate simple networks that illustrate the improved performance of LON over CNN on simple tasks for estimating the gradient magnitude squared, for regressing shape area and perimeter lengths, and for explainability of individual pixels’ influence on the result.

Cite this Paper

BibTeX
@InProceedings{pmlr-v265-sporring25a, title = {Locally orderless networks}, author = {Sporring, Jon and Xu, Peidi and Lu, Jiahao and Lauze, Francois Bernard and Darkner, Sune}, booktitle = {Proceedings of the 6th Northern Lights Deep Learning Conference (NLDL)}, pages = {239--244}, year = {2025}, editor = {Lutchyn, Tetiana and Ramírez Rivera, Adín and Ricaud, Benjamin}, volume = {265}, series = {Proceedings of Machine Learning Research}, month = {07--09 Jan}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v265/main/assets/sporring25a/sporring25a.pdf}, url = {https://proceedings.mlr.press/v265/sporring25a.html}, abstract = {We present Locally Orderless Networks (LON) and the theoretical foundation that links them to Convolutional Neural Networks (CNN), Scale-space histograms, and measurement theory. The key elements are a regular sampling of the bias and the derivative of the activation function. We compare LON, CNN, and Scale-space histograms on prototypical single-layer networks. We show how LON and CNN can emulate each other and how LON expands the set of functions computable to non-linear functions such as squaring. We demonstrate simple networks that illustrate the improved performance of LON over CNN on simple tasks for estimating the gradient magnitude squared, for regressing shape area and perimeter lengths, and for explainability of individual pixels’ influence on the result.} }
Endnote
%0 Conference Paper %T Locally orderless networks %A Jon Sporring %A Peidi Xu %A Jiahao Lu %A Francois Bernard Lauze %A Sune Darkner %B Proceedings of the 6th Northern Lights Deep Learning Conference (NLDL) %C Proceedings of Machine Learning Research %D 2025 %E Tetiana Lutchyn %E Adín Ramírez Rivera %E Benjamin Ricaud %F pmlr-v265-sporring25a %I PMLR %P 239--244 %U https://proceedings.mlr.press/v265/sporring25a.html %V 265 %X We present Locally Orderless Networks (LON) and the theoretical foundation that links them to Convolutional Neural Networks (CNN), Scale-space histograms, and measurement theory. The key elements are a regular sampling of the bias and the derivative of the activation function. We compare LON, CNN, and Scale-space histograms on prototypical single-layer networks. We show how LON and CNN can emulate each other and how LON expands the set of functions computable to non-linear functions such as squaring. We demonstrate simple networks that illustrate the improved performance of LON over CNN on simple tasks for estimating the gradient magnitude squared, for regressing shape area and perimeter lengths, and for explainability of individual pixels’ influence on the result.
APA
Sporring, J., Xu, P., Lu, J., Lauze, F.B. & Darkner, S.. (2025). Locally orderless networks. Proceedings of the 6th Northern Lights Deep Learning Conference (NLDL), in Proceedings of Machine Learning Research 265:239-244 Available from https://proceedings.mlr.press/v265/sporring25a.html.

Related Material