Data-Dependent Conversion to a Compact Integer-Weighted Representation of a Weighted Voting Classifier

Mitsuki Maekawa, Atsuyoshi Nakamura, Mineichi Kudo
Proceedings of The 12th Asian Conference on Machine Learning, PMLR 129:241-256, 2020.

Abstract

We propose a method of converting a real-weighted voting classifier to a compact integer-weighted voting classifier. Real-weighted voting classifiers like those trained using boosting are very popular and widely used due to their high prediction performance. Real numbers, however, are space-consuming and its floating-point arithmetic is slow compared to integer arithmetic, so compact integer weights are preferable for implementation on devices with small computational resources. Our conversion makes use of given feature vectors and solves an integer linear programming problem that minimizes the sum of integer weights under the constraint of keeping the classification result for the vectors unchanged. According to our experimental results using datasets of UCI Machine Learning Repository, the bit representation sizes are reduced to $5.2$-$33.4$% within $3.7$% test accuracy degrade in 7 of 8 datasets for the weighted voting classifiers of decision stumps learned using AdaBoost-SAMME.

Cite this Paper

BibTeX
@InProceedings{pmlr-v129-maekawa20a, title = {Data-Dependent Conversion to a Compact Integer-Weighted Representation of a Weighted Voting Classifier}, author = {Maekawa, Mitsuki and Nakamura, Atsuyoshi and Kudo, Mineichi}, booktitle = {Proceedings of The 12th Asian Conference on Machine Learning}, pages = {241--256}, year = {2020}, editor = {Pan, Sinno Jialin and Sugiyama, Masashi}, volume = {129}, series = {Proceedings of Machine Learning Research}, month = {18--20 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v129/maekawa20a/maekawa20a.pdf}, url = {https://proceedings.mlr.press/v129/maekawa20a.html}, abstract = {We propose a method of converting a real-weighted voting classifier to a compact integer-weighted voting classifier. Real-weighted voting classifiers like those trained using boosting are very popular and widely used due to their high prediction performance. Real numbers, however, are space-consuming and its floating-point arithmetic is slow compared to integer arithmetic, so compact integer weights are preferable for implementation on devices with small computational resources. Our conversion makes use of given feature vectors and solves an integer linear programming problem that minimizes the sum of integer weights under the constraint of keeping the classification result for the vectors unchanged. According to our experimental results using datasets of UCI Machine Learning Repository, the bit representation sizes are reduced to $5.2$-$33.4$% within $3.7$% test accuracy degrade in 7 of 8 datasets for the weighted voting classifiers of decision stumps learned using AdaBoost-SAMME. } }
Endnote
%0 Conference Paper %T Data-Dependent Conversion to a Compact Integer-Weighted Representation of a Weighted Voting Classifier %A Mitsuki Maekawa %A Atsuyoshi Nakamura %A Mineichi Kudo %B Proceedings of The 12th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Sinno Jialin Pan %E Masashi Sugiyama %F pmlr-v129-maekawa20a %I PMLR %P 241--256 %U https://proceedings.mlr.press/v129/maekawa20a.html %V 129 %X We propose a method of converting a real-weighted voting classifier to a compact integer-weighted voting classifier. Real-weighted voting classifiers like those trained using boosting are very popular and widely used due to their high prediction performance. Real numbers, however, are space-consuming and its floating-point arithmetic is slow compared to integer arithmetic, so compact integer weights are preferable for implementation on devices with small computational resources. Our conversion makes use of given feature vectors and solves an integer linear programming problem that minimizes the sum of integer weights under the constraint of keeping the classification result for the vectors unchanged. According to our experimental results using datasets of UCI Machine Learning Repository, the bit representation sizes are reduced to $5.2$-$33.4$% within $3.7$% test accuracy degrade in 7 of 8 datasets for the weighted voting classifiers of decision stumps learned using AdaBoost-SAMME.
APA
Maekawa, M., Nakamura, A. & Kudo, M.. (2020). Data-Dependent Conversion to a Compact Integer-Weighted Representation of a Weighted Voting Classifier. Proceedings of The 12th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 129:241-256 Available from https://proceedings.mlr.press/v129/maekawa20a.html.

Related Material