Faster Activation Functions at the Edge for Post-Training Speedups

Anton Lydike, Jun Bi, Jackson Woodruff
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:83738-83750, 2026.

Abstract

On-device AI has gained significant attention for enabling efficient, low-latency inference on edge devices. However, tight resource constraints on these platforms make the deployment of accurate and lightweight deep learning models challenging. In particular, advanced activation functions (AFs) like Swish and GELU often incur high inference overhead due to the lack of hardware fast-paths for exponentiation and division, restricting edge-ML applications to simple AFs like ReLU, limiting model accuracy. To address this, we propose FFCC, a compiler that automatically generates efficient approximations of AFs through floating-point reinterpretation. These functions do not require hardware fast-paths, meaning they remain fast on edge devices, but are accurate enough to be used as post-training drop-ins. FFCC takes a specification of AFs using basic floating-point operators and applies derivation rules to lower these expressions into efficient instruction sequences. Our experiments show that FFCC provides fast approximations of AFs, achieving order-of-magnitude speed-ups over accurate baselines on Arm M7, Aarch64 and Intel platforms. Using ConvNeXt as an example, we demonstrate how these activation-level gains translate to end-to-end speed-ups, and do not result in significant loss of model accuracy.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-lydike26a, title = {Faster Activation Functions at the Edge for Post-Training Speedups}, author = {Lydike, Anton and Bi, Jun and Woodruff, Jackson}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {83738--83750}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/lydike26a/lydike26a.pdf}, url = {https://proceedings.mlr.press/v306/lydike26a.html}, abstract = {On-device AI has gained significant attention for enabling efficient, low-latency inference on edge devices. However, tight resource constraints on these platforms make the deployment of accurate and lightweight deep learning models challenging. In particular, advanced activation functions (AFs) like Swish and GELU often incur high inference overhead due to the lack of hardware fast-paths for exponentiation and division, restricting edge-ML applications to simple AFs like ReLU, limiting model accuracy. To address this, we propose FFCC, a compiler that automatically generates efficient approximations of AFs through floating-point reinterpretation. These functions do not require hardware fast-paths, meaning they remain fast on edge devices, but are accurate enough to be used as post-training drop-ins. FFCC takes a specification of AFs using basic floating-point operators and applies derivation rules to lower these expressions into efficient instruction sequences. Our experiments show that FFCC provides fast approximations of AFs, achieving order-of-magnitude speed-ups over accurate baselines on Arm M7, Aarch64 and Intel platforms. Using ConvNeXt as an example, we demonstrate how these activation-level gains translate to end-to-end speed-ups, and do not result in significant loss of model accuracy.} }
Endnote
%0 Conference Paper %T Faster Activation Functions at the Edge for Post-Training Speedups %A Anton Lydike %A Jun Bi %A Jackson Woodruff %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-lydike26a %I PMLR %P 83738--83750 %U https://proceedings.mlr.press/v306/lydike26a.html %V 306 %X On-device AI has gained significant attention for enabling efficient, low-latency inference on edge devices. However, tight resource constraints on these platforms make the deployment of accurate and lightweight deep learning models challenging. In particular, advanced activation functions (AFs) like Swish and GELU often incur high inference overhead due to the lack of hardware fast-paths for exponentiation and division, restricting edge-ML applications to simple AFs like ReLU, limiting model accuracy. To address this, we propose FFCC, a compiler that automatically generates efficient approximations of AFs through floating-point reinterpretation. These functions do not require hardware fast-paths, meaning they remain fast on edge devices, but are accurate enough to be used as post-training drop-ins. FFCC takes a specification of AFs using basic floating-point operators and applies derivation rules to lower these expressions into efficient instruction sequences. Our experiments show that FFCC provides fast approximations of AFs, achieving order-of-magnitude speed-ups over accurate baselines on Arm M7, Aarch64 and Intel platforms. Using ConvNeXt as an example, we demonstrate how these activation-level gains translate to end-to-end speed-ups, and do not result in significant loss of model accuracy.
APA
Lydike, A., Bi, J. & Woodruff, J.. (2026). Faster Activation Functions at the Edge for Post-Training Speedups. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:83738-83750 Available from https://proceedings.mlr.press/v306/lydike26a.html.

Related Material