Understanding Generalization Through Visualizations

W. Ronny Huang, Zeyad Emam, Micah Goldblum, Liam Fowl, Justin K. Terry, Furong Huang, Tom Goldstein
Proceedings on "I Can't Believe It's Not Better!" at NeurIPS Workshops, PMLR 137:87-97, 2020.

Abstract

The power of neural networks lies in their ability to generalize to unseen data, yet the underlying reasons for this phenomenon remain elusive. Numerous rigorous attempts have been made to explain generalization, but available bounds are still quite loose, and analysis does not always lead to true understanding. The goal of this work is to make generalization more intuitive. Using visualization methods, we discuss the mystery of generalization, the geometry of loss landscapes, and how the curse (or, rather, the blessing) of dimensionality causes optimizers to settle into minima that generalize well.

Cite this Paper

BibTeX
@InProceedings{pmlr-v137-huang20a, title = {Understanding Generalization Through Visualizations}, author = {Huang, W. Ronny and Emam, Zeyad and Goldblum, Micah and Fowl, Liam and Terry, Justin K. and Huang, Furong and Goldstein, Tom}, booktitle = {Proceedings on "I Can't Believe It's Not Better!" at NeurIPS Workshops}, pages = {87--97}, year = {2020}, editor = {Zosa Forde, Jessica and Ruiz, Francisco and Pradier, Melanie F. and Schein, Aaron}, volume = {137}, series = {Proceedings of Machine Learning Research}, month = {12 Dec}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v137/huang20a/huang20a.pdf}, url = {https://proceedings.mlr.press/v137/huang20a.html}, abstract = {The power of neural networks lies in their ability to generalize to unseen data, yet the underlying reasons for this phenomenon remain elusive. Numerous rigorous attempts have been made to explain generalization, but available bounds are still quite loose, and analysis does not always lead to true understanding. The goal of this work is to make generalization more intuitive. Using visualization methods, we discuss the mystery of generalization, the geometry of loss landscapes, and how the curse (or, rather, the blessing) of dimensionality causes optimizers to settle into minima that generalize well.} }
Endnote
%0 Conference Paper %T Understanding Generalization Through Visualizations %A W. Ronny Huang %A Zeyad Emam %A Micah Goldblum %A Liam Fowl %A Justin K. Terry %A Furong Huang %A Tom Goldstein %B Proceedings on "I Can't Believe It's Not Better!" at NeurIPS Workshops %C Proceedings of Machine Learning Research %D 2020 %E Jessica Zosa Forde %E Francisco Ruiz %E Melanie F. Pradier %E Aaron Schein %F pmlr-v137-huang20a %I PMLR %P 87--97 %U https://proceedings.mlr.press/v137/huang20a.html %V 137 %X The power of neural networks lies in their ability to generalize to unseen data, yet the underlying reasons for this phenomenon remain elusive. Numerous rigorous attempts have been made to explain generalization, but available bounds are still quite loose, and analysis does not always lead to true understanding. The goal of this work is to make generalization more intuitive. Using visualization methods, we discuss the mystery of generalization, the geometry of loss landscapes, and how the curse (or, rather, the blessing) of dimensionality causes optimizers to settle into minima that generalize well.
APA
Huang, W.R., Emam, Z., Goldblum, M., Fowl, L., Terry, J.K., Huang, F. & Goldstein, T.. (2020). Understanding Generalization Through Visualizations. Proceedings on "I Can't Believe It's Not Better!" at NeurIPS Workshops, in Proceedings of Machine Learning Research 137:87-97 Available from https://proceedings.mlr.press/v137/huang20a.html.

Related Material