Freeze, Diffuse, Decode: Task-Aware Adaptation of Transformer Embeddings for Antimicrobial Peptide Design

Pankhil Gawade, Adam Izdebski, Myriam Lizotte, Kevin R. Moon, Jake Slater Rhodes, Guy Wolf, Ewa Szczurek
Proceedings of the 2nd Conference on Topology, Algebra, and Geometry in Data Science(TAG-DS 2026), PMLR 334(2):190-210, 2026.

Abstract

Pretrained transformers provide general-purpose molecular embeddings for downstream tasks. While these embeddings provide task-agnostic structural patterns, they lack task-specific alignment, limiting downstream performance. Here, we introduce Freeze, Diffuse, Decode (FDD), a diffusion-based framework that adapts pretrained transformer embeddings to downstream tasks by building a task supervised diffusion geometry over the frozen embeddings, without any backbone training. Applied to antimicrobial peptide design, FDD yields low-dimensional, predictive, and interpretable representations that support property prediction, retrieval, and latent-space interpolation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v334-gawade26a, title = {Freeze, Diffuse, Decode: Task-Aware Adaptation of Transformer Embeddings for Antimicrobial Peptide Design}, author = {Gawade, Pankhil and Izdebski, Adam and Lizotte, Myriam and Moon, Kevin R. and Rhodes, Jake Slater and Wolf, Guy and Szczurek, Ewa}, booktitle = {Proceedings of the 2nd Conference on Topology, Algebra, and Geometry in Data Science(TAG-DS 2026)}, pages = {190--210}, year = {2026}, editor = {Berman, Eddie and Bernárdez, Guillermo and Chen, Samantha and Cloninger, Alex and Doster, Timothy and Emerson, Tegan and Grigsby, J. Elisenda and Kvinge, Henry and Lawrence, Hannah and Marrinan, Tim and Myers, Audun and Papillon, Mathilde and Tahmasebi, Behrooz and Telyatnikov, Lev and Walters, Robin and Weber, Melanie and Xie, YuQing and Yeats, Eric}, volume = {334}, number = {2}, series = {Proceedings of Machine Learning Research}, month = {18--20 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v334/main/assets/gawade26a/gawade26a.pdf}, url = {https://proceedings.mlr.press/v334/gawade26a.html}, abstract = {Pretrained transformers provide general-purpose molecular embeddings for downstream tasks. While these embeddings provide task-agnostic structural patterns, they lack task-specific alignment, limiting downstream performance. Here, we introduce Freeze, Diffuse, Decode (FDD), a diffusion-based framework that adapts pretrained transformer embeddings to downstream tasks by building a task supervised diffusion geometry over the frozen embeddings, without any backbone training. Applied to antimicrobial peptide design, FDD yields low-dimensional, predictive, and interpretable representations that support property prediction, retrieval, and latent-space interpolation.} }
Endnote
%0 Conference Paper %T Freeze, Diffuse, Decode: Task-Aware Adaptation of Transformer Embeddings for Antimicrobial Peptide Design %A Pankhil Gawade %A Adam Izdebski %A Myriam Lizotte %A Kevin R. Moon %A Jake Slater Rhodes %A Guy Wolf %A Ewa Szczurek %B Proceedings of the 2nd Conference on Topology, Algebra, and Geometry in Data Science(TAG-DS 2026) %C Proceedings of Machine Learning Research %D 2026 %E Eddie Berman %E Guillermo Bernárdez %E Samantha Chen %E Alex Cloninger %E Timothy Doster %E Tegan Emerson %E J. Elisenda Grigsby %E Henry Kvinge %E Hannah Lawrence %E Tim Marrinan %E Audun Myers %E Mathilde Papillon %E Behrooz Tahmasebi %E Lev Telyatnikov %E Robin Walters %E Melanie Weber %E YuQing Xie %E Eric Yeats %F pmlr-v334-gawade26a %I PMLR %P 190--210 %U https://proceedings.mlr.press/v334/gawade26a.html %V 334 %N 2 %X Pretrained transformers provide general-purpose molecular embeddings for downstream tasks. While these embeddings provide task-agnostic structural patterns, they lack task-specific alignment, limiting downstream performance. Here, we introduce Freeze, Diffuse, Decode (FDD), a diffusion-based framework that adapts pretrained transformer embeddings to downstream tasks by building a task supervised diffusion geometry over the frozen embeddings, without any backbone training. Applied to antimicrobial peptide design, FDD yields low-dimensional, predictive, and interpretable representations that support property prediction, retrieval, and latent-space interpolation.
APA
Gawade, P., Izdebski, A., Lizotte, M., Moon, K.R., Rhodes, J.S., Wolf, G. & Szczurek, E.. (2026). Freeze, Diffuse, Decode: Task-Aware Adaptation of Transformer Embeddings for Antimicrobial Peptide Design. Proceedings of the 2nd Conference on Topology, Algebra, and Geometry in Data Science(TAG-DS 2026), in Proceedings of Machine Learning Research 334(2):190-210 Available from https://proceedings.mlr.press/v334/gawade26a.html.

Related Material