Amortized In-Context Mixed Effect Transformer Models: A Zero-Shot Approach for Pharmacokinetics

Cesar Ojeda, Ramses J Sanchez, Wilhelm Huisinga, Niklas Hartung
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:5248-5256, 2026.

Abstract

Accurate dose-response forecasting under sparse sampling is central to precision pharmacotherapy. We present the Amortized In-Context Mixed-Effect Transformer (AICMET) model, a transformer-based, latent-variable framework that unifies mechanistic compartmental priors with amortized, in-context Bayesian inference. AICMET is \emph{pre-trained} on hundreds of thousands of synthetic pharmacokinetic trajectories with Ornstein-Uhlenbeck priors over the parameters of compartment models, endowing the model with strong inductive biases and enabling \emph{zero-shot adaptation} to new compounds. At inference time, AICMET is \emph{conditioned on the collective context of previously profiled trial participants}, generating calibrated posterior predictions for newly enrolled patients after a few early drug concentration measurements. This capability collapses traditional model development cycles from weeks to seconds, while preserving some degree of expert modelling. Experiments across public datasets show that AICMET attains state-of-the-art predictive accuracy, and faithfully quantifies inter-patient variability - outperforming both nonlinear mixed-effects baselines and recent neural ODE variants. Our code repository, pretrained model and tutorials are available online.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-ojeda26a, title = { Amortized In-Context Mixed Effect Transformer Models: A Zero-Shot Approach for Pharmacokinetics }, author = {Ojeda, Cesar and Sanchez, Ramses J and Huisinga, Wilhelm and Hartung, Niklas}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {5248--5256}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/ojeda26a/ojeda26a.pdf}, url = {https://proceedings.mlr.press/v300/ojeda26a.html}, abstract = { Accurate dose-response forecasting under sparse sampling is central to precision pharmacotherapy. We present the Amortized In-Context Mixed-Effect Transformer (AICMET) model, a transformer-based, latent-variable framework that unifies mechanistic compartmental priors with amortized, in-context Bayesian inference. AICMET is \emph{pre-trained} on hundreds of thousands of synthetic pharmacokinetic trajectories with Ornstein-Uhlenbeck priors over the parameters of compartment models, endowing the model with strong inductive biases and enabling \emph{zero-shot adaptation} to new compounds. At inference time, AICMET is \emph{conditioned on the collective context of previously profiled trial participants}, generating calibrated posterior predictions for newly enrolled patients after a few early drug concentration measurements. This capability collapses traditional model development cycles from weeks to seconds, while preserving some degree of expert modelling. Experiments across public datasets show that AICMET attains state-of-the-art predictive accuracy, and faithfully quantifies inter-patient variability - outperforming both nonlinear mixed-effects baselines and recent neural ODE variants. Our code repository, pretrained model and tutorials are available online. } }
Endnote
%0 Conference Paper %T Amortized In-Context Mixed Effect Transformer Models: A Zero-Shot Approach for Pharmacokinetics %A Cesar Ojeda %A Ramses J Sanchez %A Wilhelm Huisinga %A Niklas Hartung %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-ojeda26a %I PMLR %P 5248--5256 %U https://proceedings.mlr.press/v300/ojeda26a.html %V 300 %X Accurate dose-response forecasting under sparse sampling is central to precision pharmacotherapy. We present the Amortized In-Context Mixed-Effect Transformer (AICMET) model, a transformer-based, latent-variable framework that unifies mechanistic compartmental priors with amortized, in-context Bayesian inference. AICMET is \emph{pre-trained} on hundreds of thousands of synthetic pharmacokinetic trajectories with Ornstein-Uhlenbeck priors over the parameters of compartment models, endowing the model with strong inductive biases and enabling \emph{zero-shot adaptation} to new compounds. At inference time, AICMET is \emph{conditioned on the collective context of previously profiled trial participants}, generating calibrated posterior predictions for newly enrolled patients after a few early drug concentration measurements. This capability collapses traditional model development cycles from weeks to seconds, while preserving some degree of expert modelling. Experiments across public datasets show that AICMET attains state-of-the-art predictive accuracy, and faithfully quantifies inter-patient variability - outperforming both nonlinear mixed-effects baselines and recent neural ODE variants. Our code repository, pretrained model and tutorials are available online.
APA
Ojeda, C., Sanchez, R.J., Huisinga, W. & Hartung, N.. (2026). Amortized In-Context Mixed Effect Transformer Models: A Zero-Shot Approach for Pharmacokinetics . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:5248-5256 Available from https://proceedings.mlr.press/v300/ojeda26a.html.

Related Material