Lag Operator SSMs: A Geometric Framework for Structured State Space Modeling

Sutashu Tomonaga, Kenji Doya, Noboru Murata
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:2647-2655, 2026.

Abstract

Structured State Space Models (SSMs), which are at the heart of the recently popular Mamba architecture, are powerful tools for sequence modeling. However, their theoretical foundation relies on a complex, multi-stage process of continuous-time modeling and subsequent discretization, which can obscure intuition. We introduce a direct, first-principles framework for constructing discrete-time SSMs that is both flexible and modular. Our approach is based on a novel lag operator, which geometrically derives the discrete-time recurrence by measuring how the system’s basis functions undergo what we call a \emph{domain expansion} from one timestep to the next. The resulting state matrices are computed via a single inner product involving this operator, enabling a modular design space for creating novel SSMs by flexibly combining different basis functions and time-warping schemes. To validate our framework, we demonstrate that a specific instance exactly recovers the recurrence of the influential HiPPO model. Numerical simulations confirm our derivation, providing new theoretical tools for designing flexible and robust sequence models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-tomonaga26a, title = { Lag Operator SSMs: A Geometric Framework for Structured State Space Modeling }, author = {Tomonaga, Sutashu and Doya, Kenji and Murata, Noboru}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {2647--2655}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/tomonaga26a/tomonaga26a.pdf}, url = {https://proceedings.mlr.press/v300/tomonaga26a.html}, abstract = { Structured State Space Models (SSMs), which are at the heart of the recently popular Mamba architecture, are powerful tools for sequence modeling. However, their theoretical foundation relies on a complex, multi-stage process of continuous-time modeling and subsequent discretization, which can obscure intuition. We introduce a direct, first-principles framework for constructing discrete-time SSMs that is both flexible and modular. Our approach is based on a novel lag operator, which geometrically derives the discrete-time recurrence by measuring how the system’s basis functions undergo what we call a \emph{domain expansion} from one timestep to the next. The resulting state matrices are computed via a single inner product involving this operator, enabling a modular design space for creating novel SSMs by flexibly combining different basis functions and time-warping schemes. To validate our framework, we demonstrate that a specific instance exactly recovers the recurrence of the influential HiPPO model. Numerical simulations confirm our derivation, providing new theoretical tools for designing flexible and robust sequence models. } }
Endnote
%0 Conference Paper %T Lag Operator SSMs: A Geometric Framework for Structured State Space Modeling %A Sutashu Tomonaga %A Kenji Doya %A Noboru Murata %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-tomonaga26a %I PMLR %P 2647--2655 %U https://proceedings.mlr.press/v300/tomonaga26a.html %V 300 %X Structured State Space Models (SSMs), which are at the heart of the recently popular Mamba architecture, are powerful tools for sequence modeling. However, their theoretical foundation relies on a complex, multi-stage process of continuous-time modeling and subsequent discretization, which can obscure intuition. We introduce a direct, first-principles framework for constructing discrete-time SSMs that is both flexible and modular. Our approach is based on a novel lag operator, which geometrically derives the discrete-time recurrence by measuring how the system’s basis functions undergo what we call a \emph{domain expansion} from one timestep to the next. The resulting state matrices are computed via a single inner product involving this operator, enabling a modular design space for creating novel SSMs by flexibly combining different basis functions and time-warping schemes. To validate our framework, we demonstrate that a specific instance exactly recovers the recurrence of the influential HiPPO model. Numerical simulations confirm our derivation, providing new theoretical tools for designing flexible and robust sequence models.
APA
Tomonaga, S., Doya, K. & Murata, N.. (2026). Lag Operator SSMs: A Geometric Framework for Structured State Space Modeling . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:2647-2655 Available from https://proceedings.mlr.press/v300/tomonaga26a.html.

Related Material