MKD: Multi-Knowledge Distillation for Real-Time Object Detection on Edge Devices

Yanjun Xu, Chunqi Tian, Xuhui Xia, Yaoru Sun
Proceedings of the 17th Asian Conference on Machine Learning, PMLR 304:590-605, 2025.

Abstract

Real-time object detection on resource-constrained edge devices presents a significant challenge in balancing performance and efficiency. This paper introduces a novel knowledge distillation framework designed to enhance the capabilities of lightweight student models for object detection tasks. Our approach, Multi-Scale Frequency-Aware Distillation (MSFAD), integrates three key components: multi-scale distillation, frequency domain mask distillation, and feature alignment distillation. Multi-scale distillation enables the student to learn feature representations at various levels of granularity. Frequency domain mask distillation improves the student’s ability to focus on relevant regions. Feature alignment distillation facilitates the transfer of channel-wise knowledge from teacher to student. We combine these techniques with a traditional detection loss to form a comprehensive loss function, balanced by a hyperparameter $\\alpha$. Experimental results across various scenarios demonstrate that MSFAD significantly improves detection accuracy while reducing computational and storage costs. Our approach clearly presents significant performance gains and faster inference speeds.

Cite this Paper

BibTeX
@InProceedings{pmlr-v304-xu25b, title = {MKD: Multi-Knowledge Distillation for Real-Time Object Detection on Edge Devices}, author = {Xu, Yanjun and Tian, Chunqi and Xia, Xuhui and Sun, Yaoru}, booktitle = {Proceedings of the 17th Asian Conference on Machine Learning}, pages = {590--605}, year = {2025}, editor = {Lee, Hung-yi and Liu, Tongliang}, volume = {304}, series = {Proceedings of Machine Learning Research}, month = {09--12 Dec}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v304/main/assets/xu25b/xu25b.pdf}, url = {https://proceedings.mlr.press/v304/xu25b.html}, abstract = {Real-time object detection on resource-constrained edge devices presents a significant challenge in balancing performance and efficiency. This paper introduces a novel knowledge distillation framework designed to enhance the capabilities of lightweight student models for object detection tasks. Our approach, Multi-Scale Frequency-Aware Distillation (MSFAD), integrates three key components: multi-scale distillation, frequency domain mask distillation, and feature alignment distillation. Multi-scale distillation enables the student to learn feature representations at various levels of granularity. Frequency domain mask distillation improves the student’s ability to focus on relevant regions. Feature alignment distillation facilitates the transfer of channel-wise knowledge from teacher to student. We combine these techniques with a traditional detection loss to form a comprehensive loss function, balanced by a hyperparameter $\\alpha$. Experimental results across various scenarios demonstrate that MSFAD significantly improves detection accuracy while reducing computational and storage costs. Our approach clearly presents significant performance gains and faster inference speeds.} }
Endnote
%0 Conference Paper %T MKD: Multi-Knowledge Distillation for Real-Time Object Detection on Edge Devices %A Yanjun Xu %A Chunqi Tian %A Xuhui Xia %A Yaoru Sun %B Proceedings of the 17th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Hung-yi Lee %E Tongliang Liu %F pmlr-v304-xu25b %I PMLR %P 590--605 %U https://proceedings.mlr.press/v304/xu25b.html %V 304 %X Real-time object detection on resource-constrained edge devices presents a significant challenge in balancing performance and efficiency. This paper introduces a novel knowledge distillation framework designed to enhance the capabilities of lightweight student models for object detection tasks. Our approach, Multi-Scale Frequency-Aware Distillation (MSFAD), integrates three key components: multi-scale distillation, frequency domain mask distillation, and feature alignment distillation. Multi-scale distillation enables the student to learn feature representations at various levels of granularity. Frequency domain mask distillation improves the student’s ability to focus on relevant regions. Feature alignment distillation facilitates the transfer of channel-wise knowledge from teacher to student. We combine these techniques with a traditional detection loss to form a comprehensive loss function, balanced by a hyperparameter $\\alpha$. Experimental results across various scenarios demonstrate that MSFAD significantly improves detection accuracy while reducing computational and storage costs. Our approach clearly presents significant performance gains and faster inference speeds.
APA
Xu, Y., Tian, C., Xia, X. & Sun, Y.. (2025). MKD: Multi-Knowledge Distillation for Real-Time Object Detection on Edge Devices. Proceedings of the 17th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 304:590-605 Available from https://proceedings.mlr.press/v304/xu25b.html.

Related Material