Hypernetwork-based Implicit Posterior Estimation and Model Averaging of CNN

Kenya Ukai, Takashi Matsubara, Kuniaki Uehara
Proceedings of The 10th Asian Conference on Machine Learning, PMLR 95:176-191, 2018.

Abstract

Deep neural networks have a rich ability to learn complex representations and achieved remarkable results in various tasks. However, they are prone to overfitting due to the limited number of training samples; regularizing the learning process of neural networks is critical. In this paper, we propose a novel regularization method, which estimates parameters of a large convolutional neural network as implicit probabilistic distributions generated by a hypernetwork. Also, we can perform model averaging to improve the network performance. Experimental results demonstrate our regularization method outperformed the commonly-used maximum a posterior (MAP) estimation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v95-ukai18a, title = {Hypernetwork-based Implicit Posterior Estimation and Model Averaging of CNN}, author = {Ukai, Kenya and Matsubara, Takashi and Uehara, Kuniaki}, booktitle = {Proceedings of The 10th Asian Conference on Machine Learning}, pages = {176--191}, year = {2018}, editor = {Zhu, Jun and Takeuchi, Ichiro}, volume = {95}, series = {Proceedings of Machine Learning Research}, month = {14--16 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v95/ukai18a/ukai18a.pdf}, url = {https://proceedings.mlr.press/v95/ukai18a.html}, abstract = {Deep neural networks have a rich ability to learn complex representations and achieved remarkable results in various tasks. However, they are prone to overfitting due to the limited number of training samples; regularizing the learning process of neural networks is critical. In this paper, we propose a novel regularization method, which estimates parameters of a large convolutional neural network as implicit probabilistic distributions generated by a hypernetwork. Also, we can perform model averaging to improve the network performance. Experimental results demonstrate our regularization method outperformed the commonly-used maximum a posterior (MAP) estimation.} }
Endnote
%0 Conference Paper %T Hypernetwork-based Implicit Posterior Estimation and Model Averaging of CNN %A Kenya Ukai %A Takashi Matsubara %A Kuniaki Uehara %B Proceedings of The 10th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jun Zhu %E Ichiro Takeuchi %F pmlr-v95-ukai18a %I PMLR %P 176--191 %U https://proceedings.mlr.press/v95/ukai18a.html %V 95 %X Deep neural networks have a rich ability to learn complex representations and achieved remarkable results in various tasks. However, they are prone to overfitting due to the limited number of training samples; regularizing the learning process of neural networks is critical. In this paper, we propose a novel regularization method, which estimates parameters of a large convolutional neural network as implicit probabilistic distributions generated by a hypernetwork. Also, we can perform model averaging to improve the network performance. Experimental results demonstrate our regularization method outperformed the commonly-used maximum a posterior (MAP) estimation.
APA
Ukai, K., Matsubara, T. & Uehara, K.. (2018). Hypernetwork-based Implicit Posterior Estimation and Model Averaging of CNN. Proceedings of The 10th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 95:176-191 Available from https://proceedings.mlr.press/v95/ukai18a.html.

Related Material