One-Shot Learning with a Hierarchical Nonparametric Bayesian Model

Ruslan Salakhutdinov, Joshua Tenenbaum, Antonio Torralba
Proceedings of ICML Workshop on Unsupervised and Transfer Learning, PMLR 27:195-206, 2012.

Abstract

We develop a hierarchical Bayesian model that learns categories from single training examples. The model transfers acquired knowledge from previously learned categories to a novel category, in the form of a prior over category means and variances. The model discovers how to group categories into meaningful super-categories that express different priors for new classes. Given a single example of a novel category, we can efficiently infer which super-category the novel category belongs to, and thereby estimate not only the new categories mean but also an appropriate similarity metric based on parameters inherited from the super-category. On MNIST and MSR Cambridge image datasets the model learns useful representations of novel categories based on just a single training example, and performs significantly better than simpler hierarchical Bayesian approaches. It can also discover new categories in a completely unsupervised fashion, given just one or a few examples.

Cite this Paper

BibTeX
@InProceedings{pmlr-v27-salakhutdinov12a, title = {One-Shot Learning with a Hierarchical Nonparametric Bayesian Model}, author = {Salakhutdinov, Ruslan and Tenenbaum, Joshua and Torralba, Antonio}, booktitle = {Proceedings of ICML Workshop on Unsupervised and Transfer Learning}, pages = {195--206}, year = {2012}, editor = {Guyon, Isabelle and Dror, Gideon and Lemaire, Vincent and Taylor, Graham and Silver, Daniel}, volume = {27}, series = {Proceedings of Machine Learning Research}, address = {Bellevue, Washington, USA}, month = {02 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v27/salakhutdinov12a/salakhutdinov12a.pdf}, url = {https://proceedings.mlr.press/v27/salakhutdinov12a.html}, abstract = {We develop a hierarchical Bayesian model that learns categories from single training examples. The model transfers acquired knowledge from previously learned categories to a novel category, in the form of a prior over category means and variances. The model discovers how to group categories into meaningful super-categories that express different priors for new classes. Given a single example of a novel category, we can efficiently infer which super-category the novel category belongs to, and thereby estimate not only the new categories mean but also an appropriate similarity metric based on parameters inherited from the super-category. On MNIST and MSR Cambridge image datasets the model learns useful representations of novel categories based on just a single training example, and performs significantly better than simpler hierarchical Bayesian approaches. It can also discover new categories in a completely unsupervised fashion, given just one or a few examples.} }
Endnote
%0 Conference Paper %T One-Shot Learning with a Hierarchical Nonparametric Bayesian Model %A Ruslan Salakhutdinov %A Joshua Tenenbaum %A Antonio Torralba %B Proceedings of ICML Workshop on Unsupervised and Transfer Learning %C Proceedings of Machine Learning Research %D 2012 %E Isabelle Guyon %E Gideon Dror %E Vincent Lemaire %E Graham Taylor %E Daniel Silver %F pmlr-v27-salakhutdinov12a %I PMLR %P 195--206 %U https://proceedings.mlr.press/v27/salakhutdinov12a.html %V 27 %X We develop a hierarchical Bayesian model that learns categories from single training examples. The model transfers acquired knowledge from previously learned categories to a novel category, in the form of a prior over category means and variances. The model discovers how to group categories into meaningful super-categories that express different priors for new classes. Given a single example of a novel category, we can efficiently infer which super-category the novel category belongs to, and thereby estimate not only the new categories mean but also an appropriate similarity metric based on parameters inherited from the super-category. On MNIST and MSR Cambridge image datasets the model learns useful representations of novel categories based on just a single training example, and performs significantly better than simpler hierarchical Bayesian approaches. It can also discover new categories in a completely unsupervised fashion, given just one or a few examples.
RIS
TY - CPAPER TI - One-Shot Learning with a Hierarchical Nonparametric Bayesian Model AU - Ruslan Salakhutdinov AU - Joshua Tenenbaum AU - Antonio Torralba BT - Proceedings of ICML Workshop on Unsupervised and Transfer Learning DA - 2012/06/27 ED - Isabelle Guyon ED - Gideon Dror ED - Vincent Lemaire ED - Graham Taylor ED - Daniel Silver ID - pmlr-v27-salakhutdinov12a PB - PMLR DP - Proceedings of Machine Learning Research VL - 27 SP - 195 EP - 206 L1 - http://proceedings.mlr.press/v27/salakhutdinov12a/salakhutdinov12a.pdf UR - https://proceedings.mlr.press/v27/salakhutdinov12a.html AB - We develop a hierarchical Bayesian model that learns categories from single training examples. The model transfers acquired knowledge from previously learned categories to a novel category, in the form of a prior over category means and variances. The model discovers how to group categories into meaningful super-categories that express different priors for new classes. Given a single example of a novel category, we can efficiently infer which super-category the novel category belongs to, and thereby estimate not only the new categories mean but also an appropriate similarity metric based on parameters inherited from the super-category. On MNIST and MSR Cambridge image datasets the model learns useful representations of novel categories based on just a single training example, and performs significantly better than simpler hierarchical Bayesian approaches. It can also discover new categories in a completely unsupervised fashion, given just one or a few examples. ER -
APA
Salakhutdinov, R., Tenenbaum, J. & Torralba, A.. (2012). One-Shot Learning with a Hierarchical Nonparametric Bayesian Model. Proceedings of ICML Workshop on Unsupervised and Transfer Learning, in Proceedings of Machine Learning Research 27:195-206 Available from https://proceedings.mlr.press/v27/salakhutdinov12a.html.

Related Material