A Comparative Empirical Study of Relative Embedding Alignment in Neural Dynamical System Forecasters

Deniz Kucukahmetler, Maximilian Jean Hemmann, Julian Mosig von Aehrenfeld, Maximilian Amthor, Christian Deubel, Nico Scherf, Diaaeldin Taha
Proceedings of the Geometry, Topology, and Machine Learning Workshop, PMLR 325:201-214, 2026.

Abstract

We study neural forecasters for dynamical systems through the lens of representational alignment. We introduce anchor-based, geometry-agnostic \emph{relative embeddings} that remove rotational and scaling ambiguities, enabling robust cross-seed and cross-architecture comparison. Across diverse periodic, quasi-periodic, and chaotic systems, we observe consistent family-level patterns: MLPs align with MLPs, RNNs with RNNs, and ESNs show reduced alignment on chaotic dynamics, while transformers often align weakly but still perform well. Alignment generally correlates with forecasting accuracy, yet high accuracy can coexist with low alignment. Relative embeddings thus offer a simple, reproducible basis for comparing learned dynamics. \footnote{This workshop paper is a condensed companion to the extended version published in Transactions on Machine Learning Research (TMLR); \citep{kucukahmetler2026relative}. We thank the Max Planck Computing and Data Facility (MPCDF) for providing GPU resources. D.K. is supported by BMFTR in DAAD project 57616814 (SECAI). N.S. is supported by BMFTR (Federal Ministry of Research, Technology and Space) through ACONITE (16IS22065) and the Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI.) Leipzig and by the European Union and the Free State of Saxony through BIOWIN.}

Cite this Paper

BibTeX
@InProceedings{pmlr-v325-kucukahmetler26a, title = {A Comparative Empirical Study of Relative Embedding Alignment in Neural Dynamical System Forecasters}, author = {Kucukahmetler, Deniz and Hemmann, Maximilian Jean and Mosig von Aehrenfeld, Julian and Amthor, Maximilian and Deubel, Christian and Scherf, Nico and Taha, Diaaeldin}, booktitle = {Proceedings of the Geometry, Topology, and Machine Learning Workshop}, pages = {201--214}, year = {2026}, editor = {Bleher, Michael and Jensen, Freya and Maier, Levin and Taha, Diaaeldin and Wienhard, Anna}, volume = {325}, series = {Proceedings of Machine Learning Research}, month = {10--14 Nov}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v325/main/assets/kucukahmetler26a/kucukahmetler26a.pdf}, url = {https://proceedings.mlr.press/v325/kucukahmetler26a.html}, abstract = {We study neural forecasters for dynamical systems through the lens of representational alignment. We introduce anchor-based, geometry-agnostic \emph{relative embeddings} that remove rotational and scaling ambiguities, enabling robust cross-seed and cross-architecture comparison. Across diverse periodic, quasi-periodic, and chaotic systems, we observe consistent family-level patterns: MLPs align with MLPs, RNNs with RNNs, and ESNs show reduced alignment on chaotic dynamics, while transformers often align weakly but still perform well. Alignment generally correlates with forecasting accuracy, yet high accuracy can coexist with low alignment. Relative embeddings thus offer a simple, reproducible basis for comparing learned dynamics. \footnote{This workshop paper is a condensed companion to the extended version published in Transactions on Machine Learning Research (TMLR); \citep{kucukahmetler2026relative}. We thank the Max Planck Computing and Data Facility (MPCDF) for providing GPU resources. D.K. is supported by BMFTR in DAAD project 57616814 (SECAI). N.S. is supported by BMFTR (Federal Ministry of Research, Technology and Space) through ACONITE (16IS22065) and the Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI.) Leipzig and by the European Union and the Free State of Saxony through BIOWIN.}} }
Endnote
%0 Conference Paper %T A Comparative Empirical Study of Relative Embedding Alignment in Neural Dynamical System Forecasters %A Deniz Kucukahmetler %A Maximilian Jean Hemmann %A Julian Mosig von Aehrenfeld %A Maximilian Amthor %A Christian Deubel %A Nico Scherf %A Diaaeldin Taha %B Proceedings of the Geometry, Topology, and Machine Learning Workshop %C Proceedings of Machine Learning Research %D 2026 %E Michael Bleher %E Freya Jensen %E Levin Maier %E Diaaeldin Taha %E Anna Wienhard %F pmlr-v325-kucukahmetler26a %I PMLR %P 201--214 %U https://proceedings.mlr.press/v325/kucukahmetler26a.html %V 325 %X We study neural forecasters for dynamical systems through the lens of representational alignment. We introduce anchor-based, geometry-agnostic \emph{relative embeddings} that remove rotational and scaling ambiguities, enabling robust cross-seed and cross-architecture comparison. Across diverse periodic, quasi-periodic, and chaotic systems, we observe consistent family-level patterns: MLPs align with MLPs, RNNs with RNNs, and ESNs show reduced alignment on chaotic dynamics, while transformers often align weakly but still perform well. Alignment generally correlates with forecasting accuracy, yet high accuracy can coexist with low alignment. Relative embeddings thus offer a simple, reproducible basis for comparing learned dynamics. \footnote{This workshop paper is a condensed companion to the extended version published in Transactions on Machine Learning Research (TMLR); \citep{kucukahmetler2026relative}. We thank the Max Planck Computing and Data Facility (MPCDF) for providing GPU resources. D.K. is supported by BMFTR in DAAD project 57616814 (SECAI). N.S. is supported by BMFTR (Federal Ministry of Research, Technology and Space) through ACONITE (16IS22065) and the Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI.) Leipzig and by the European Union and the Free State of Saxony through BIOWIN.}
APA
Kucukahmetler, D., Hemmann, M.J., Mosig von Aehrenfeld, J., Amthor, M., Deubel, C., Scherf, N. & Taha, D.. (2026). A Comparative Empirical Study of Relative Embedding Alignment in Neural Dynamical System Forecasters. Proceedings of the Geometry, Topology, and Machine Learning Workshop, in Proceedings of Machine Learning Research 325:201-214 Available from https://proceedings.mlr.press/v325/kucukahmetler26a.html.

Related Material