Byzantine-tolerant distributed multiclass sparse linear discriminant analysis

Yajie Bao, Weidong Liu, Xiaojun Mao, Weijia Xiong
Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, PMLR 180:129-138, 2022.

Abstract

Communication cost and security issues are both important in large-scale distributed machine learning. In this paper, we investigate a multiclass sparse classification problem under two distributed systems. We propose two distributed multiclass sparse discriminant analysis algorithms based on mean-aggregation and median-aggregation under the normal distributed system or Byzantine failure system. Both of them are computation and communication efficient. Several theoretical results, including estimation error bounds, and support recovery, are established. With moderate initial estimators, our iterative estimators achieve a (near-)optimal rate and exact support recovery after a constant number of rounds. Experiments on both synthetic and real datasets are provided to demonstrate the effectiveness of our proposed methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v180-bao22b, title = {Byzantine-tolerant distributed multiclass sparse linear discriminant analysis}, author = {Bao, Yajie and Liu, Weidong and Mao, Xiaojun and Xiong, Weijia}, booktitle = {Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence}, pages = {129--138}, year = {2022}, editor = {Cussens, James and Zhang, Kun}, volume = {180}, series = {Proceedings of Machine Learning Research}, month = {01--05 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v180/bao22b/bao22b.pdf}, url = {https://proceedings.mlr.press/v180/bao22b.html}, abstract = {Communication cost and security issues are both important in large-scale distributed machine learning. In this paper, we investigate a multiclass sparse classification problem under two distributed systems. We propose two distributed multiclass sparse discriminant analysis algorithms based on mean-aggregation and median-aggregation under the normal distributed system or Byzantine failure system. Both of them are computation and communication efficient. Several theoretical results, including estimation error bounds, and support recovery, are established. With moderate initial estimators, our iterative estimators achieve a (near-)optimal rate and exact support recovery after a constant number of rounds. Experiments on both synthetic and real datasets are provided to demonstrate the effectiveness of our proposed methods.} }
Endnote
%0 Conference Paper %T Byzantine-tolerant distributed multiclass sparse linear discriminant analysis %A Yajie Bao %A Weidong Liu %A Xiaojun Mao %A Weijia Xiong %B Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2022 %E James Cussens %E Kun Zhang %F pmlr-v180-bao22b %I PMLR %P 129--138 %U https://proceedings.mlr.press/v180/bao22b.html %V 180 %X Communication cost and security issues are both important in large-scale distributed machine learning. In this paper, we investigate a multiclass sparse classification problem under two distributed systems. We propose two distributed multiclass sparse discriminant analysis algorithms based on mean-aggregation and median-aggregation under the normal distributed system or Byzantine failure system. Both of them are computation and communication efficient. Several theoretical results, including estimation error bounds, and support recovery, are established. With moderate initial estimators, our iterative estimators achieve a (near-)optimal rate and exact support recovery after a constant number of rounds. Experiments on both synthetic and real datasets are provided to demonstrate the effectiveness of our proposed methods.
APA
Bao, Y., Liu, W., Mao, X. & Xiong, W.. (2022). Byzantine-tolerant distributed multiclass sparse linear discriminant analysis. Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 180:129-138 Available from https://proceedings.mlr.press/v180/bao22b.html.

Related Material