Identifiability of Priors from Bounded Sample Sizes with Applications to Transfer Learning

Liu Yang, Steve Hanneke, Jaime Carbonell
Proceedings of the 24th Annual Conference on Learning Theory, PMLR 19:789-806, 2011.

Abstract

We explore a transfer learning setting, in which a finite sequence of target concepts are sampled independently with an unknown distribution from a known family. We study the total number of labeled examples required to learn all targets to an arbitrary specified expected accuracy, focusing on the asymptotics in the number of tasks and the desired accuracy. Our primary interest is formally understanding the fundamental benefits of transfer learning, compared to learning each target independently from the others. Our approach to the transfer problem is general, in the sense that it can be used with a variety of learning protocols. The key insight driving our approach is that the distribution of the target concepts is identifiable from the joint distribution over a number of random labeled data points equal the Vapnik-Chervonenkis dimension of the concept space. This is not necessarily the case for the joint distribution over any smaller number of points. This work has particularly interesting implications when applied to active learning methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v19-yang11a, title = {Identifiability of Priors from Bounded Sample Sizes with Applications to Transfer Learning}, author = {Yang, Liu and Hanneke, Steve and Carbonell, Jaime}, booktitle = {Proceedings of the 24th Annual Conference on Learning Theory}, pages = {789--806}, year = {2011}, editor = {Kakade, Sham M. and von Luxburg, Ulrike}, volume = {19}, series = {Proceedings of Machine Learning Research}, address = {Budapest, Hungary}, month = {09--11 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v19/yang11a/yang11a.pdf}, url = {https://proceedings.mlr.press/v19/yang11a.html}, abstract = {We explore a transfer learning setting, in which a finite sequence of target concepts are sampled independently with an unknown distribution from a known family. We study the total number of labeled examples required to learn all targets to an arbitrary specified expected accuracy, focusing on the asymptotics in the number of tasks and the desired accuracy. Our primary interest is formally understanding the fundamental benefits of transfer learning, compared to learning each target independently from the others. Our approach to the transfer problem is general, in the sense that it can be used with a variety of learning protocols. The key insight driving our approach is that the distribution of the target concepts is identifiable from the joint distribution over a number of random labeled data points equal the Vapnik-Chervonenkis dimension of the concept space. This is not necessarily the case for the joint distribution over any smaller number of points. This work has particularly interesting implications when applied to active learning methods.} }
Endnote
%0 Conference Paper %T Identifiability of Priors from Bounded Sample Sizes with Applications to Transfer Learning %A Liu Yang %A Steve Hanneke %A Jaime Carbonell %B Proceedings of the 24th Annual Conference on Learning Theory %C Proceedings of Machine Learning Research %D 2011 %E Sham M. Kakade %E Ulrike von Luxburg %F pmlr-v19-yang11a %I PMLR %P 789--806 %U https://proceedings.mlr.press/v19/yang11a.html %V 19 %X We explore a transfer learning setting, in which a finite sequence of target concepts are sampled independently with an unknown distribution from a known family. We study the total number of labeled examples required to learn all targets to an arbitrary specified expected accuracy, focusing on the asymptotics in the number of tasks and the desired accuracy. Our primary interest is formally understanding the fundamental benefits of transfer learning, compared to learning each target independently from the others. Our approach to the transfer problem is general, in the sense that it can be used with a variety of learning protocols. The key insight driving our approach is that the distribution of the target concepts is identifiable from the joint distribution over a number of random labeled data points equal the Vapnik-Chervonenkis dimension of the concept space. This is not necessarily the case for the joint distribution over any smaller number of points. This work has particularly interesting implications when applied to active learning methods.
RIS
TY - CPAPER TI - Identifiability of Priors from Bounded Sample Sizes with Applications to Transfer Learning AU - Liu Yang AU - Steve Hanneke AU - Jaime Carbonell BT - Proceedings of the 24th Annual Conference on Learning Theory DA - 2011/12/21 ED - Sham M. Kakade ED - Ulrike von Luxburg ID - pmlr-v19-yang11a PB - PMLR DP - Proceedings of Machine Learning Research VL - 19 SP - 789 EP - 806 L1 - http://proceedings.mlr.press/v19/yang11a/yang11a.pdf UR - https://proceedings.mlr.press/v19/yang11a.html AB - We explore a transfer learning setting, in which a finite sequence of target concepts are sampled independently with an unknown distribution from a known family. We study the total number of labeled examples required to learn all targets to an arbitrary specified expected accuracy, focusing on the asymptotics in the number of tasks and the desired accuracy. Our primary interest is formally understanding the fundamental benefits of transfer learning, compared to learning each target independently from the others. Our approach to the transfer problem is general, in the sense that it can be used with a variety of learning protocols. The key insight driving our approach is that the distribution of the target concepts is identifiable from the joint distribution over a number of random labeled data points equal the Vapnik-Chervonenkis dimension of the concept space. This is not necessarily the case for the joint distribution over any smaller number of points. This work has particularly interesting implications when applied to active learning methods. ER -
APA
Yang, L., Hanneke, S. & Carbonell, J.. (2011). Identifiability of Priors from Bounded Sample Sizes with Applications to Transfer Learning. Proceedings of the 24th Annual Conference on Learning Theory, in Proceedings of Machine Learning Research 19:789-806 Available from https://proceedings.mlr.press/v19/yang11a.html.

Related Material