PSForest: Improving Deep Forest via Feature Pooling and Error Screening

Shiwen Ni, Hung-Yu Kao
Proceedings of The 12th Asian Conference on Machine Learning, PMLR 129:769-781, 2020.

Abstract

In recent years, most of the research on deep learning is based on deep neural networks, which uses the backpropagation algorithm to train parameters of nonlinear layers. Recently, a non-NN style deep model called Deep Forest or gcForest was proposed by Zhou and Feng, which is a deep learning model based on random forests and the training process does not rely on backpropagation. In this paper, we propose PSForest, which can be regarded as a modification of the standard Deep Forest. The main idea for improving the efficiency and performance of the Deep Forest is to do multi-grained pooling of raw features and screening the class vector of each layer based on out-of-bag error. The experiment on different datasets shows that our proposed model achieves predictive accuracy comparable to or better than gcForest, with lower memory requirement and smaller time cost. The study significantly improves the competitiveness of deep forests, further demonstrating that deep learning is more than just deep neural networks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v129-ni20a, title = {PSForest: Improving Deep Forest via Feature Pooling and Error Screening}, author = {Ni, Shiwen and Kao, Hung-Yu}, booktitle = {Proceedings of The 12th Asian Conference on Machine Learning}, pages = {769--781}, year = {2020}, editor = {Pan, Sinno Jialin and Sugiyama, Masashi}, volume = {129}, series = {Proceedings of Machine Learning Research}, month = {18--20 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v129/ni20a/ni20a.pdf}, url = {https://proceedings.mlr.press/v129/ni20a.html}, abstract = {In recent years, most of the research on deep learning is based on deep neural networks, which uses the backpropagation algorithm to train parameters of nonlinear layers. Recently, a non-NN style deep model called Deep Forest or gcForest was proposed by Zhou and Feng, which is a deep learning model based on random forests and the training process does not rely on backpropagation. In this paper, we propose PSForest, which can be regarded as a modification of the standard Deep Forest. The main idea for improving the efficiency and performance of the Deep Forest is to do multi-grained pooling of raw features and screening the class vector of each layer based on out-of-bag error. The experiment on different datasets shows that our proposed model achieves predictive accuracy comparable to or better than gcForest, with lower memory requirement and smaller time cost. The study significantly improves the competitiveness of deep forests, further demonstrating that deep learning is more than just deep neural networks.} }
Endnote
%0 Conference Paper %T PSForest: Improving Deep Forest via Feature Pooling and Error Screening %A Shiwen Ni %A Hung-Yu Kao %B Proceedings of The 12th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Sinno Jialin Pan %E Masashi Sugiyama %F pmlr-v129-ni20a %I PMLR %P 769--781 %U https://proceedings.mlr.press/v129/ni20a.html %V 129 %X In recent years, most of the research on deep learning is based on deep neural networks, which uses the backpropagation algorithm to train parameters of nonlinear layers. Recently, a non-NN style deep model called Deep Forest or gcForest was proposed by Zhou and Feng, which is a deep learning model based on random forests and the training process does not rely on backpropagation. In this paper, we propose PSForest, which can be regarded as a modification of the standard Deep Forest. The main idea for improving the efficiency and performance of the Deep Forest is to do multi-grained pooling of raw features and screening the class vector of each layer based on out-of-bag error. The experiment on different datasets shows that our proposed model achieves predictive accuracy comparable to or better than gcForest, with lower memory requirement and smaller time cost. The study significantly improves the competitiveness of deep forests, further demonstrating that deep learning is more than just deep neural networks.
APA
Ni, S. & Kao, H.. (2020). PSForest: Improving Deep Forest via Feature Pooling and Error Screening. Proceedings of The 12th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 129:769-781 Available from https://proceedings.mlr.press/v129/ni20a.html.

Related Material