Convex Invariance Learning

Tony Jebara
Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, PMLR R4:149-156, 2003.

Abstract

Invariance and representation learning are important precursors to modeling and classification tools particularly for non-Euclidean spaces such as images, strings and nonvectorial data. This article proposes a method for learning invariances in data while jointly estimating a model. The technique results in a convex programming problem with a consistent and unique solution. Representation variables are considered as affine transformations confined by multiple equality and inequality constraints. These interact individually with each datum yet maintain the overall solvability of the model estimation process while uniquely solving for the representational variables themselves. The method is applicable to various types of modeling, including maximum likelihood estimation, principal components analysis, and discriminative methods. Starting from affine invariance, several types of invariances are proposed and implemented as convex programs including clustering, permutation, selection, rotation, and translation. Experiments on non-vectorial data such as images and collections of tuples provide promising results.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR4-jebara03a, title = {Convex Invariance Learning}, author = {Jebara, Tony}, booktitle = {Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics}, pages = {149--156}, year = {2003}, editor = {Bishop, Christopher M. and Frey, Brendan J.}, volume = {R4}, series = {Proceedings of Machine Learning Research}, month = {03--06 Jan}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/r4/jebara03a/jebara03a.pdf}, url = {https://proceedings.mlr.press/r4/jebara03a.html}, abstract = {Invariance and representation learning are important precursors to modeling and classification tools particularly for non-Euclidean spaces such as images, strings and nonvectorial data. This article proposes a method for learning invariances in data while jointly estimating a model. The technique results in a convex programming problem with a consistent and unique solution. Representation variables are considered as affine transformations confined by multiple equality and inequality constraints. These interact individually with each datum yet maintain the overall solvability of the model estimation process while uniquely solving for the representational variables themselves. The method is applicable to various types of modeling, including maximum likelihood estimation, principal components analysis, and discriminative methods. Starting from affine invariance, several types of invariances are proposed and implemented as convex programs including clustering, permutation, selection, rotation, and translation. Experiments on non-vectorial data such as images and collections of tuples provide promising results.}, note = {Reissued by PMLR on 01 April 2021.} }
Endnote
%0 Conference Paper %T Convex Invariance Learning %A Tony Jebara %B Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2003 %E Christopher M. Bishop %E Brendan J. Frey %F pmlr-vR4-jebara03a %I PMLR %P 149--156 %U https://proceedings.mlr.press/r4/jebara03a.html %V R4 %X Invariance and representation learning are important precursors to modeling and classification tools particularly for non-Euclidean spaces such as images, strings and nonvectorial data. This article proposes a method for learning invariances in data while jointly estimating a model. The technique results in a convex programming problem with a consistent and unique solution. Representation variables are considered as affine transformations confined by multiple equality and inequality constraints. These interact individually with each datum yet maintain the overall solvability of the model estimation process while uniquely solving for the representational variables themselves. The method is applicable to various types of modeling, including maximum likelihood estimation, principal components analysis, and discriminative methods. Starting from affine invariance, several types of invariances are proposed and implemented as convex programs including clustering, permutation, selection, rotation, and translation. Experiments on non-vectorial data such as images and collections of tuples provide promising results. %Z Reissued by PMLR on 01 April 2021.
APA
Jebara, T.. (2003). Convex Invariance Learning. Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research R4:149-156 Available from https://proceedings.mlr.press/r4/jebara03a.html. Reissued by PMLR on 01 April 2021.

Related Material