Deep Structured Energy Based Models for Anomaly Detection

Shuangfei Zhai, Yu Cheng, Weining Lu, Zhongfei Zhang
Proceedings of The 33rd International Conference on Machine Learning, PMLR 48:1100-1109, 2016.

Abstract

In this paper, we attack the anomaly detection problem by directly modeling the data distribution with deep architectures. We hence propose deep structured energy based models (DSEBMs), where the energy function is the output of a deterministic deep neural network with structure. We develop novel model architectures to integrate EBMs with different types of data such as static data, sequential data, and spatial data, and apply appropriate model architectures to adapt to the data structure. Our training algorithm is built upon the recent development of score matching (Hyvarinen, 2005), which connects an EBM with a regularized autoencoder, eliminating the need for complicated sampling method. Statistically sound decision criterion can be derived for anomaly detection purpose from the perspective of the energy landscape of the data distribution. We investigate two decision criteria for performing anomaly detection: the energy score and the reconstruction error. Extensive empirical studies on benchmark anomaly detection tasks demonstrate that our proposed model consistently matches or outperforms all the competing methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v48-zhai16, title = {Deep Structured Energy Based Models for Anomaly Detection}, author = {Zhai, Shuangfei and Cheng, Yu and Lu, Weining and Zhang, Zhongfei}, booktitle = {Proceedings of The 33rd International Conference on Machine Learning}, pages = {1100--1109}, year = {2016}, editor = {Balcan, Maria Florina and Weinberger, Kilian Q.}, volume = {48}, series = {Proceedings of Machine Learning Research}, address = {New York, New York, USA}, month = {20--22 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v48/zhai16.pdf}, url = {https://proceedings.mlr.press/v48/zhai16.html}, abstract = {In this paper, we attack the anomaly detection problem by directly modeling the data distribution with deep architectures. We hence propose deep structured energy based models (DSEBMs), where the energy function is the output of a deterministic deep neural network with structure. We develop novel model architectures to integrate EBMs with different types of data such as static data, sequential data, and spatial data, and apply appropriate model architectures to adapt to the data structure. Our training algorithm is built upon the recent development of score matching (Hyvarinen, 2005), which connects an EBM with a regularized autoencoder, eliminating the need for complicated sampling method. Statistically sound decision criterion can be derived for anomaly detection purpose from the perspective of the energy landscape of the data distribution. We investigate two decision criteria for performing anomaly detection: the energy score and the reconstruction error. Extensive empirical studies on benchmark anomaly detection tasks demonstrate that our proposed model consistently matches or outperforms all the competing methods.} }
Endnote
%0 Conference Paper %T Deep Structured Energy Based Models for Anomaly Detection %A Shuangfei Zhai %A Yu Cheng %A Weining Lu %A Zhongfei Zhang %B Proceedings of The 33rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2016 %E Maria Florina Balcan %E Kilian Q. Weinberger %F pmlr-v48-zhai16 %I PMLR %P 1100--1109 %U https://proceedings.mlr.press/v48/zhai16.html %V 48 %X In this paper, we attack the anomaly detection problem by directly modeling the data distribution with deep architectures. We hence propose deep structured energy based models (DSEBMs), where the energy function is the output of a deterministic deep neural network with structure. We develop novel model architectures to integrate EBMs with different types of data such as static data, sequential data, and spatial data, and apply appropriate model architectures to adapt to the data structure. Our training algorithm is built upon the recent development of score matching (Hyvarinen, 2005), which connects an EBM with a regularized autoencoder, eliminating the need for complicated sampling method. Statistically sound decision criterion can be derived for anomaly detection purpose from the perspective of the energy landscape of the data distribution. We investigate two decision criteria for performing anomaly detection: the energy score and the reconstruction error. Extensive empirical studies on benchmark anomaly detection tasks demonstrate that our proposed model consistently matches or outperforms all the competing methods.
RIS
TY - CPAPER TI - Deep Structured Energy Based Models for Anomaly Detection AU - Shuangfei Zhai AU - Yu Cheng AU - Weining Lu AU - Zhongfei Zhang BT - Proceedings of The 33rd International Conference on Machine Learning DA - 2016/06/11 ED - Maria Florina Balcan ED - Kilian Q. Weinberger ID - pmlr-v48-zhai16 PB - PMLR DP - Proceedings of Machine Learning Research VL - 48 SP - 1100 EP - 1109 L1 - http://proceedings.mlr.press/v48/zhai16.pdf UR - https://proceedings.mlr.press/v48/zhai16.html AB - In this paper, we attack the anomaly detection problem by directly modeling the data distribution with deep architectures. We hence propose deep structured energy based models (DSEBMs), where the energy function is the output of a deterministic deep neural network with structure. We develop novel model architectures to integrate EBMs with different types of data such as static data, sequential data, and spatial data, and apply appropriate model architectures to adapt to the data structure. Our training algorithm is built upon the recent development of score matching (Hyvarinen, 2005), which connects an EBM with a regularized autoencoder, eliminating the need for complicated sampling method. Statistically sound decision criterion can be derived for anomaly detection purpose from the perspective of the energy landscape of the data distribution. We investigate two decision criteria for performing anomaly detection: the energy score and the reconstruction error. Extensive empirical studies on benchmark anomaly detection tasks demonstrate that our proposed model consistently matches or outperforms all the competing methods. ER -
APA
Zhai, S., Cheng, Y., Lu, W. & Zhang, Z.. (2016). Deep Structured Energy Based Models for Anomaly Detection. Proceedings of The 33rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 48:1100-1109 Available from https://proceedings.mlr.press/v48/zhai16.html.

Related Material