Be Like Water: Adaptive Floating Point for Machine Learning

Thomas Yeh, Max Sterner, Zerlina Lai, Brandon Chuang, Alexander Ihler
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:25490-25500, 2022.

Abstract

In the pursuit of optimizing memory and compute density to accelerate machine learning applications, reduced precision training and inference has been an active area of research. While some approaches selectively apply low precision computations, this may require costly off-chip data transfers or mixed precision support. In this paper, we propose a novel numerical representation, Adaptive Floating Point (AFP), that dynamically adjusts to the characteristics of deep learning data. AFP requires no changes to the model topology, requires no additional training, and applies to all layers of DNN models. We evaluate AFP on a spectrum of representative models in computer vision and NLP, and show that our technique enables ultra-low precision inference of deep learning models while providing accuracy comparable to full precision inference. By dynamically adjusting to ML data, AFP increases memory density by 1.6x, 1.6x, and 3.2x and compute density by 4x, 1.3x, and 12x when compared to BFP, BFloat16, and FP32.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-yeh22a, title = {Be Like Water: Adaptive Floating Point for Machine Learning}, author = {Yeh, Thomas and Sterner, Max and Lai, Zerlina and Chuang, Brandon and Ihler, Alexander}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {25490--25500}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/yeh22a/yeh22a.pdf}, url = {https://proceedings.mlr.press/v162/yeh22a.html}, abstract = {In the pursuit of optimizing memory and compute density to accelerate machine learning applications, reduced precision training and inference has been an active area of research. While some approaches selectively apply low precision computations, this may require costly off-chip data transfers or mixed precision support. In this paper, we propose a novel numerical representation, Adaptive Floating Point (AFP), that dynamically adjusts to the characteristics of deep learning data. AFP requires no changes to the model topology, requires no additional training, and applies to all layers of DNN models. We evaluate AFP on a spectrum of representative models in computer vision and NLP, and show that our technique enables ultra-low precision inference of deep learning models while providing accuracy comparable to full precision inference. By dynamically adjusting to ML data, AFP increases memory density by 1.6x, 1.6x, and 3.2x and compute density by 4x, 1.3x, and 12x when compared to BFP, BFloat16, and FP32.} }
Endnote
%0 Conference Paper %T Be Like Water: Adaptive Floating Point for Machine Learning %A Thomas Yeh %A Max Sterner %A Zerlina Lai %A Brandon Chuang %A Alexander Ihler %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-yeh22a %I PMLR %P 25490--25500 %U https://proceedings.mlr.press/v162/yeh22a.html %V 162 %X In the pursuit of optimizing memory and compute density to accelerate machine learning applications, reduced precision training and inference has been an active area of research. While some approaches selectively apply low precision computations, this may require costly off-chip data transfers or mixed precision support. In this paper, we propose a novel numerical representation, Adaptive Floating Point (AFP), that dynamically adjusts to the characteristics of deep learning data. AFP requires no changes to the model topology, requires no additional training, and applies to all layers of DNN models. We evaluate AFP on a spectrum of representative models in computer vision and NLP, and show that our technique enables ultra-low precision inference of deep learning models while providing accuracy comparable to full precision inference. By dynamically adjusting to ML data, AFP increases memory density by 1.6x, 1.6x, and 3.2x and compute density by 4x, 1.3x, and 12x when compared to BFP, BFloat16, and FP32.
APA
Yeh, T., Sterner, M., Lai, Z., Chuang, B. & Ihler, A.. (2022). Be Like Water: Adaptive Floating Point for Machine Learning. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:25490-25500 Available from https://proceedings.mlr.press/v162/yeh22a.html.

Related Material