Using Deep Neural Networks to Automate Large Scale Statistical Analysis for Big Data Applications

Rongrong Zhang, Wei Deng, Michael Yu Zhu
Proceedings of the Ninth Asian Conference on Machine Learning, PMLR 77:311-326, 2017.

Abstract

Statistical analysis (SA) is a complex process to deduce population properties from analysis of data. It usually takes a well-trained analyst to successfully perform SA, and it becomes extremely challenging to apply SA to big data applications. We propose to use deep neural networks to automate the SA process. In particular, we propose to construct convolutional neural networks (CNNs) to perform automatic model selection and parameter estimation, two most important SA tasks. We refer to the resulting CNNs as the neural model selector and the neural model estimator, respectively, which can be properly trained using labeled data systematically generated from candidate models. Simulation study shows that both the selector and estimator demonstrate excellent performances. The idea and proposed framework can be further extended to automate the entire SA process and have the potential to revolutionize how SA is performed in big data analytics.

Cite this Paper

BibTeX
@InProceedings{pmlr-v77-zhang17d, title = {Using Deep Neural Networks to Automate Large Scale Statistical Analysis for Big Data Applications}, author = {Zhang, Rongrong and Deng, Wei and Zhu, Michael Yu}, booktitle = {Proceedings of the Ninth Asian Conference on Machine Learning}, pages = {311--326}, year = {2017}, editor = {Zhang, Min-Ling and Noh, Yung-Kyun}, volume = {77}, series = {Proceedings of Machine Learning Research}, address = {Yonsei University, Seoul, Republic of Korea}, month = {15--17 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v77/zhang17d/zhang17d.pdf}, url = {https://proceedings.mlr.press/v77/zhang17d.html}, abstract = {Statistical analysis (SA) is a complex process to deduce population properties from analysis of data. It usually takes a well-trained analyst to successfully perform SA, and it becomes extremely challenging to apply SA to big data applications. We propose to use deep neural networks to automate the SA process. In particular, we propose to construct convolutional neural networks (CNNs) to perform automatic model selection and parameter estimation, two most important SA tasks. We refer to the resulting CNNs as the neural model selector and the neural model estimator, respectively, which can be properly trained using labeled data systematically generated from candidate models. Simulation study shows that both the selector and estimator demonstrate excellent performances. The idea and proposed framework can be further extended to automate the entire SA process and have the potential to revolutionize how SA is performed in big data analytics.} }
Endnote
%0 Conference Paper %T Using Deep Neural Networks to Automate Large Scale Statistical Analysis for Big Data Applications %A Rongrong Zhang %A Wei Deng %A Michael Yu Zhu %B Proceedings of the Ninth Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Min-Ling Zhang %E Yung-Kyun Noh %F pmlr-v77-zhang17d %I PMLR %P 311--326 %U https://proceedings.mlr.press/v77/zhang17d.html %V 77 %X Statistical analysis (SA) is a complex process to deduce population properties from analysis of data. It usually takes a well-trained analyst to successfully perform SA, and it becomes extremely challenging to apply SA to big data applications. We propose to use deep neural networks to automate the SA process. In particular, we propose to construct convolutional neural networks (CNNs) to perform automatic model selection and parameter estimation, two most important SA tasks. We refer to the resulting CNNs as the neural model selector and the neural model estimator, respectively, which can be properly trained using labeled data systematically generated from candidate models. Simulation study shows that both the selector and estimator demonstrate excellent performances. The idea and proposed framework can be further extended to automate the entire SA process and have the potential to revolutionize how SA is performed in big data analytics.
APA
Zhang, R., Deng, W. & Zhu, M.Y.. (2017). Using Deep Neural Networks to Automate Large Scale Statistical Analysis for Big Data Applications. Proceedings of the Ninth Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 77:311-326 Available from https://proceedings.mlr.press/v77/zhang17d.html.

Related Material