Variational Sparse Coding

Francesco Tonolini, Bjørn Sand Jensen, Roderick Murray-Smith
Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, PMLR 115:690-700, 2020.

Abstract

Unsupervised discovery of interpretable features and controllable generation with high-dimensional data are currently major challenges in machine learning, with applications in data visualisation, clustering and artificial data synthesis. We propose a model based on variational auto-encoders (VAEs) in which interpretation is induced through latent space sparsity with a mixture of Spike and Slab distributions as prior. We derive an evidence lower bound for this model and propose a specific training method for recovering disentangled features as sparse elements in latent vectors. In our experiments, we demonstrate superior disentanglement performance to standard VAE approaches when an estimate of the number of true sources of variation is not available and objects display different combinations of attributes. Furthermore, the new model provides unique capabilities, such as recovering feature exploitation, synthesising samples that share attributes with a given input object and controlling both discrete and continuous features upon generation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v115-tonolini20a, title = {Variational Sparse Coding}, author = {Tonolini, Francesco and Jensen, Bj{\o}rn Sand and Murray-Smith, Roderick}, booktitle = {Proceedings of The 35th Uncertainty in Artificial Intelligence Conference}, pages = {690--700}, year = {2020}, editor = {Adams, Ryan P. and Gogate, Vibhav}, volume = {115}, series = {Proceedings of Machine Learning Research}, month = {22--25 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v115/tonolini20a/tonolini20a.pdf}, url = {https://proceedings.mlr.press/v115/tonolini20a.html}, abstract = {Unsupervised discovery of interpretable features and controllable generation with high-dimensional data are currently major challenges in machine learning, with applications in data visualisation, clustering and artificial data synthesis. We propose a model based on variational auto-encoders (VAEs) in which interpretation is induced through latent space sparsity with a mixture of Spike and Slab distributions as prior. We derive an evidence lower bound for this model and propose a specific training method for recovering disentangled features as sparse elements in latent vectors. In our experiments, we demonstrate superior disentanglement performance to standard VAE approaches when an estimate of the number of true sources of variation is not available and objects display different combinations of attributes. Furthermore, the new model provides unique capabilities, such as recovering feature exploitation, synthesising samples that share attributes with a given input object and controlling both discrete and continuous features upon generation.} }
Endnote
%0 Conference Paper %T Variational Sparse Coding %A Francesco Tonolini %A Bjørn Sand Jensen %A Roderick Murray-Smith %B Proceedings of The 35th Uncertainty in Artificial Intelligence Conference %C Proceedings of Machine Learning Research %D 2020 %E Ryan P. Adams %E Vibhav Gogate %F pmlr-v115-tonolini20a %I PMLR %P 690--700 %U https://proceedings.mlr.press/v115/tonolini20a.html %V 115 %X Unsupervised discovery of interpretable features and controllable generation with high-dimensional data are currently major challenges in machine learning, with applications in data visualisation, clustering and artificial data synthesis. We propose a model based on variational auto-encoders (VAEs) in which interpretation is induced through latent space sparsity with a mixture of Spike and Slab distributions as prior. We derive an evidence lower bound for this model and propose a specific training method for recovering disentangled features as sparse elements in latent vectors. In our experiments, we demonstrate superior disentanglement performance to standard VAE approaches when an estimate of the number of true sources of variation is not available and objects display different combinations of attributes. Furthermore, the new model provides unique capabilities, such as recovering feature exploitation, synthesising samples that share attributes with a given input object and controlling both discrete and continuous features upon generation.
APA
Tonolini, F., Jensen, B.S. & Murray-Smith, R.. (2020). Variational Sparse Coding. Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, in Proceedings of Machine Learning Research 115:690-700 Available from https://proceedings.mlr.press/v115/tonolini20a.html.

Related Material