Reforming Generative Autoencoders via Goodness-of-Fit Hypothesis Testing

Aaron Palmer, Dipak Dey, Jinbo Bi
Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, PMLR R16:1008-1018, 2018.

Abstract

Generative models, while not new, have taken the deep learning field by storm. However, the widely used training methods have not exploited the substantial statistical literature concerning parametric distributional testing. Having sound theoretical foundations, these goodness-of-fit tests enable parts of the black box to be stripped away. In this paper we use the Shapiro-Wilk and propose a new multivari- ate generalization of Shapiro-Wilk to respec- tively test for univariate and multivariate nor- mality of the code layer of a generative autoen- coder. By replacing the discriminator in tradi- tional deep models with the hypothesis tests, we gain several advantages: objectively evalu- ate whether the encoder is actually embedding data onto a normal manifold, accurately define when convergence happens, explicitly balance between reconstruction and encoding training. Not only does our method produce competitive results, but it does so in a fraction of the time. We highlight the fact that the hypothesis tests used in our model asymptotically lead to the same solution of the L2-Wasserstein distance metrics used by several generative models to- day.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR16-palmer18a, title = {Reforming Generative Autoencoders via Goodness-of-Fit Hypothesis Testing}, author = {Palmer, Aaron and Dey, Dipak and Bi, Jinbo}, booktitle = {Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence}, pages = {1008--1018}, year = {2018}, editor = {Globerson, Amir and Silva, Ricardo}, volume = {R16}, series = {Proceedings of Machine Learning Research}, month = {06--10 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r16/main/assets/palmer18a/palmer18a.pdf}, url = {https://proceedings.mlr.press/r16/palmer18a.html}, abstract = {Generative models, while not new, have taken the deep learning field by storm. However, the widely used training methods have not exploited the substantial statistical literature concerning parametric distributional testing. Having sound theoretical foundations, these goodness-of-fit tests enable parts of the black box to be stripped away. In this paper we use the Shapiro-Wilk and propose a new multivari- ate generalization of Shapiro-Wilk to respec- tively test for univariate and multivariate nor- mality of the code layer of a generative autoen- coder. By replacing the discriminator in tradi- tional deep models with the hypothesis tests, we gain several advantages: objectively evalu- ate whether the encoder is actually embedding data onto a normal manifold, accurately define when convergence happens, explicitly balance between reconstruction and encoding training. Not only does our method produce competitive results, but it does so in a fraction of the time. We highlight the fact that the hypothesis tests used in our model asymptotically lead to the same solution of the L2-Wasserstein distance metrics used by several generative models to- day.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Reforming Generative Autoencoders via Goodness-of-Fit Hypothesis Testing %A Aaron Palmer %A Dipak Dey %A Jinbo Bi %B Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2018 %E Amir Globerson %E Ricardo Silva %F pmlr-vR16-palmer18a %I PMLR %P 1008--1018 %U https://proceedings.mlr.press/r16/palmer18a.html %V R16 %X Generative models, while not new, have taken the deep learning field by storm. However, the widely used training methods have not exploited the substantial statistical literature concerning parametric distributional testing. Having sound theoretical foundations, these goodness-of-fit tests enable parts of the black box to be stripped away. In this paper we use the Shapiro-Wilk and propose a new multivari- ate generalization of Shapiro-Wilk to respec- tively test for univariate and multivariate nor- mality of the code layer of a generative autoen- coder. By replacing the discriminator in tradi- tional deep models with the hypothesis tests, we gain several advantages: objectively evalu- ate whether the encoder is actually embedding data onto a normal manifold, accurately define when convergence happens, explicitly balance between reconstruction and encoding training. Not only does our method produce competitive results, but it does so in a fraction of the time. We highlight the fact that the hypothesis tests used in our model asymptotically lead to the same solution of the L2-Wasserstein distance metrics used by several generative models to- day. %Z Reissued by PMLR on 04 October 2026.
APA
Palmer, A., Dey, D. & Bi, J.. (2018). Reforming Generative Autoencoders via Goodness-of-Fit Hypothesis Testing. Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R16:1008-1018 Available from https://proceedings.mlr.press/r16/palmer18a.html. Reissued by PMLR on 04 October 2026.

Related Material