Sequential Disentanglement by Extracting Static Information From A Single Sequence Element

Nimrod Berman, Ilan Naiman, Idan Arbiv, Gal Fadlon, Omri Azencot
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:3539-3564, 2024.

Abstract

One of the fundamental representation learning tasks is unsupervised sequential disentanglement, where latent codes of inputs are decomposed to a single static factor and a sequence of dynamic factors. To extract this latent information, existing methods condition the static and dynamic codes on the entire input sequence. Unfortunately, these models often suffer from information leakage, i.e., the dynamic vectors encode both static and dynamic information, or vice versa, leading to a non-disentangled representation. Attempts to alleviate this problem via reducing the dynamic dimension and auxiliary loss terms gain only partial success. Instead, we propose a novel and simple architecture that mitigates information leakage by offering a simple and effective subtraction inductive bias while conditioning on a single sample. Remarkably, the resulting variational framework is simpler in terms of required loss terms, hyper-parameters, and data augmentation. We evaluate our method on multiple data-modality benchmarks including general time series, video, and audio, and we show beyond state-of-the-art results on generation and prediction tasks in comparison to several strong baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-berman24a, title = {Sequential Disentanglement by Extracting Static Information From A Single Sequence Element}, author = {Berman, Nimrod and Naiman, Ilan and Arbiv, Idan and Fadlon, Gal and Azencot, Omri}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {3539--3564}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/berman24a/berman24a.pdf}, url = {https://proceedings.mlr.press/v235/berman24a.html}, abstract = {One of the fundamental representation learning tasks is unsupervised sequential disentanglement, where latent codes of inputs are decomposed to a single static factor and a sequence of dynamic factors. To extract this latent information, existing methods condition the static and dynamic codes on the entire input sequence. Unfortunately, these models often suffer from information leakage, i.e., the dynamic vectors encode both static and dynamic information, or vice versa, leading to a non-disentangled representation. Attempts to alleviate this problem via reducing the dynamic dimension and auxiliary loss terms gain only partial success. Instead, we propose a novel and simple architecture that mitigates information leakage by offering a simple and effective subtraction inductive bias while conditioning on a single sample. Remarkably, the resulting variational framework is simpler in terms of required loss terms, hyper-parameters, and data augmentation. We evaluate our method on multiple data-modality benchmarks including general time series, video, and audio, and we show beyond state-of-the-art results on generation and prediction tasks in comparison to several strong baselines.} }
Endnote
%0 Conference Paper %T Sequential Disentanglement by Extracting Static Information From A Single Sequence Element %A Nimrod Berman %A Ilan Naiman %A Idan Arbiv %A Gal Fadlon %A Omri Azencot %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-berman24a %I PMLR %P 3539--3564 %U https://proceedings.mlr.press/v235/berman24a.html %V 235 %X One of the fundamental representation learning tasks is unsupervised sequential disentanglement, where latent codes of inputs are decomposed to a single static factor and a sequence of dynamic factors. To extract this latent information, existing methods condition the static and dynamic codes on the entire input sequence. Unfortunately, these models often suffer from information leakage, i.e., the dynamic vectors encode both static and dynamic information, or vice versa, leading to a non-disentangled representation. Attempts to alleviate this problem via reducing the dynamic dimension and auxiliary loss terms gain only partial success. Instead, we propose a novel and simple architecture that mitigates information leakage by offering a simple and effective subtraction inductive bias while conditioning on a single sample. Remarkably, the resulting variational framework is simpler in terms of required loss terms, hyper-parameters, and data augmentation. We evaluate our method on multiple data-modality benchmarks including general time series, video, and audio, and we show beyond state-of-the-art results on generation and prediction tasks in comparison to several strong baselines.
APA
Berman, N., Naiman, I., Arbiv, I., Fadlon, G. & Azencot, O.. (2024). Sequential Disentanglement by Extracting Static Information From A Single Sequence Element. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:3539-3564 Available from https://proceedings.mlr.press/v235/berman24a.html.

Related Material