SPARK: Lightweight Adaptation of Healthcare Foundation Models via Steering Knowledge Circuits

Bing Li, Qiang Fu, Yuwei Long, Huiyuan Yang
Proceedings of the 11th Machine Learning for Healthcare Conference, PMLR 340:985-1012, 2026.

Abstract

Large-scale foundation models (FMs) have demonstrated immense potential in medical time-series analysis, yet their static, “one-size-fits-all" nature limits their efficacy in patient-specific clinical settings. Traditional adaptation paradigms, such as full fine-tuning or standard parameter-efficient fine-tuning (PEFT), are computationally prohibitive for edge-device deployment and often overfit when faced with the chronic data scarcity and long-tail distributions characteristic of healthcare. In this work, we reveal that pre-trained medical FMs inherently possess sparse, multi-layer “Physiological Knowledge Circuits" dedicated to representing distinct clinical patterns. Leveraging this mechanistic insight, we propose SPARK (Steering Personalized Adaptation via Routing Knowledge), a novel, ultra-lightweight adaptation framework. Instead of updating the model’s global weights, SPARK utilizes a Multi-Layer Personalized Steering Hub (M-PSH) to surgically inject targeted perturbations into the hidden state activations of these specific knowledge circuits. Extensive experiments across multiple clinical ECG benchmarks demonstrate that SPARK achieves a superior parameter-to-performance ratio and excels in extreme data-scarce environments, yielding an approximately 6.8% accuracy improvement in 1-shot adaptation scenarios. Furthermore, by modulating internal representations rather than relearning them from scratch, SPARK significantly enhances model robustness against rare diseases, paving the way for privacy-preserving, dynamic, and on-device personalization in continuous health monitoring.

Cite this Paper

BibTeX
@InProceedings{pmlr-v340-li26a, title = {SPARK: Lightweight Adaptation of Healthcare Foundation Models via Steering Knowledge Circuits}, author = {Li, Bing and Fu, Qiang and Long, Yuwei and Yang, Huiyuan}, booktitle = {Proceedings of the 11th Machine Learning for Healthcare Conference}, pages = {985--1012}, year = {2026}, editor = {Krishnan, Rahul G. and van Amsterdam, Wouter A. C. and Chopra, Sumit and Overgaard, Shauna and Hughes, Michael and Ötleş, Erkin and Shen, Yiqiu and Shanmugam, Divya and Nayan, Madhur and Engelhard, Matthew and Fackler, Jim and Oberst, Michael}, volume = {340}, series = {Proceedings of Machine Learning Research}, month = {12--14 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v340/main/assets/li26a/li26a.pdf}, url = {https://proceedings.mlr.press/v340/li26a.html}, abstract = {Large-scale foundation models (FMs) have demonstrated immense potential in medical time-series analysis, yet their static, “one-size-fits-all" nature limits their efficacy in patient-specific clinical settings. Traditional adaptation paradigms, such as full fine-tuning or standard parameter-efficient fine-tuning (PEFT), are computationally prohibitive for edge-device deployment and often overfit when faced with the chronic data scarcity and long-tail distributions characteristic of healthcare. In this work, we reveal that pre-trained medical FMs inherently possess sparse, multi-layer “Physiological Knowledge Circuits" dedicated to representing distinct clinical patterns. Leveraging this mechanistic insight, we propose SPARK (Steering Personalized Adaptation via Routing Knowledge), a novel, ultra-lightweight adaptation framework. Instead of updating the model’s global weights, SPARK utilizes a Multi-Layer Personalized Steering Hub (M-PSH) to surgically inject targeted perturbations into the hidden state activations of these specific knowledge circuits. Extensive experiments across multiple clinical ECG benchmarks demonstrate that SPARK achieves a superior parameter-to-performance ratio and excels in extreme data-scarce environments, yielding an approximately 6.8% accuracy improvement in 1-shot adaptation scenarios. Furthermore, by modulating internal representations rather than relearning them from scratch, SPARK significantly enhances model robustness against rare diseases, paving the way for privacy-preserving, dynamic, and on-device personalization in continuous health monitoring.} }
Endnote
%0 Conference Paper %T SPARK: Lightweight Adaptation of Healthcare Foundation Models via Steering Knowledge Circuits %A Bing Li %A Qiang Fu %A Yuwei Long %A Huiyuan Yang %B Proceedings of the 11th Machine Learning for Healthcare Conference %C Proceedings of Machine Learning Research %D 2026 %E Rahul G. Krishnan %E Wouter A. C. van Amsterdam %E Sumit Chopra %E Shauna Overgaard %E Michael Hughes %E Erkin Ötleş %E Yiqiu Shen %E Divya Shanmugam %E Madhur Nayan %E Matthew Engelhard %E Jim Fackler %E Michael Oberst %F pmlr-v340-li26a %I PMLR %P 985--1012 %U https://proceedings.mlr.press/v340/li26a.html %V 340 %X Large-scale foundation models (FMs) have demonstrated immense potential in medical time-series analysis, yet their static, “one-size-fits-all" nature limits their efficacy in patient-specific clinical settings. Traditional adaptation paradigms, such as full fine-tuning or standard parameter-efficient fine-tuning (PEFT), are computationally prohibitive for edge-device deployment and often overfit when faced with the chronic data scarcity and long-tail distributions characteristic of healthcare. In this work, we reveal that pre-trained medical FMs inherently possess sparse, multi-layer “Physiological Knowledge Circuits" dedicated to representing distinct clinical patterns. Leveraging this mechanistic insight, we propose SPARK (Steering Personalized Adaptation via Routing Knowledge), a novel, ultra-lightweight adaptation framework. Instead of updating the model’s global weights, SPARK utilizes a Multi-Layer Personalized Steering Hub (M-PSH) to surgically inject targeted perturbations into the hidden state activations of these specific knowledge circuits. Extensive experiments across multiple clinical ECG benchmarks demonstrate that SPARK achieves a superior parameter-to-performance ratio and excels in extreme data-scarce environments, yielding an approximately 6.8% accuracy improvement in 1-shot adaptation scenarios. Furthermore, by modulating internal representations rather than relearning them from scratch, SPARK significantly enhances model robustness against rare diseases, paving the way for privacy-preserving, dynamic, and on-device personalization in continuous health monitoring.
APA
Li, B., Fu, Q., Long, Y. & Yang, H.. (2026). SPARK: Lightweight Adaptation of Healthcare Foundation Models via Steering Knowledge Circuits. Proceedings of the 11th Machine Learning for Healthcare Conference, in Proceedings of Machine Learning Research 340:985-1012 Available from https://proceedings.mlr.press/v340/li26a.html.

Related Material