Neural Fisher Discriminant Analysis: Optimal Neural Network Embeddings in Polynomial Time

Burak Bartan, Mert Pilanci
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:1647-1663, 2022.

Abstract

Fisher’s Linear Discriminant Analysis (FLDA) is a statistical analysis method that linearly embeds data points to a lower dimensional space to maximize a discrimination criterion such that the variance between classes is maximized while the variance within classes is minimized. We introduce a natural extension of FLDA that employs neural networks, called Neural Fisher Discriminant Analysis (NFDA). This method finds the optimal two-layer neural network that embeds data points to optimize the same discrimination criterion. We use tools from convex optimization to transform the optimal neural network embedding problem into a convex problem. The resulting problem is easy to interpret and solve to global optimality. We evaluate the method’s performance on synthetic and real datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-bartan22a, title = {Neural {F}isher Discriminant Analysis: Optimal Neural Network Embeddings in Polynomial Time}, author = {Bartan, Burak and Pilanci, Mert}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {1647--1663}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/bartan22a/bartan22a.pdf}, url = {https://proceedings.mlr.press/v162/bartan22a.html}, abstract = {Fisher’s Linear Discriminant Analysis (FLDA) is a statistical analysis method that linearly embeds data points to a lower dimensional space to maximize a discrimination criterion such that the variance between classes is maximized while the variance within classes is minimized. We introduce a natural extension of FLDA that employs neural networks, called Neural Fisher Discriminant Analysis (NFDA). This method finds the optimal two-layer neural network that embeds data points to optimize the same discrimination criterion. We use tools from convex optimization to transform the optimal neural network embedding problem into a convex problem. The resulting problem is easy to interpret and solve to global optimality. We evaluate the method’s performance on synthetic and real datasets.} }
Endnote
%0 Conference Paper %T Neural Fisher Discriminant Analysis: Optimal Neural Network Embeddings in Polynomial Time %A Burak Bartan %A Mert Pilanci %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-bartan22a %I PMLR %P 1647--1663 %U https://proceedings.mlr.press/v162/bartan22a.html %V 162 %X Fisher’s Linear Discriminant Analysis (FLDA) is a statistical analysis method that linearly embeds data points to a lower dimensional space to maximize a discrimination criterion such that the variance between classes is maximized while the variance within classes is minimized. We introduce a natural extension of FLDA that employs neural networks, called Neural Fisher Discriminant Analysis (NFDA). This method finds the optimal two-layer neural network that embeds data points to optimize the same discrimination criterion. We use tools from convex optimization to transform the optimal neural network embedding problem into a convex problem. The resulting problem is easy to interpret and solve to global optimality. We evaluate the method’s performance on synthetic and real datasets.
APA
Bartan, B. & Pilanci, M.. (2022). Neural Fisher Discriminant Analysis: Optimal Neural Network Embeddings in Polynomial Time. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:1647-1663 Available from https://proceedings.mlr.press/v162/bartan22a.html.

Related Material