Factor Analysis with Correlated Topic Model for Multi-Modal Data

Małgorzata Łazęcka, Ewa Maria Szczurek
Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, PMLR 258:1801-1809, 2025.

Abstract

Integrating various data modalities brings valuable insights into underlying phenomena. Multimodal factor analysis (FA) uncovers shared axes of variation underlying different simple data modalities, where each sample is represented by a vector of features. However, FA is not suited for structured data modalities, such as text or single cell sequencing data, where multiple data points are measured per each sample and exhibit a clustering structure. To overcome this challenge, we introduce FACTM, a novel, multi-view and multi-structure Bayesian model that combines FA with correlated topic modeling and is optimized using variational inference. Additionally, we introduce a method for rotating latent factors to enhance interpretability with respect to binary features. On text and video benchmarks as well as real-world music and COVID-19 datasets, we demonstrate that FACTM outperforms other methods in identifying clusters in structured data, and integrating them with simple modalities via the inference of shared, interpretable factors.

Cite this Paper

BibTeX
@InProceedings{pmlr-v258-lazecka25a, title = {Factor Analysis with Correlated Topic Model for Multi-Modal Data}, author = {{\L}az{\k{e}}cka, Ma{\l}gorzata and Szczurek, Ewa Maria}, booktitle = {Proceedings of The 28th International Conference on Artificial Intelligence and Statistics}, pages = {1801--1809}, year = {2025}, editor = {Li, Yingzhen and Mandt, Stephan and Agrawal, Shipra and Khan, Emtiyaz}, volume = {258}, series = {Proceedings of Machine Learning Research}, month = {03--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v258/main/assets/lazecka25a/lazecka25a.pdf}, url = {https://proceedings.mlr.press/v258/lazecka25a.html}, abstract = {Integrating various data modalities brings valuable insights into underlying phenomena. Multimodal factor analysis (FA) uncovers shared axes of variation underlying different simple data modalities, where each sample is represented by a vector of features. However, FA is not suited for structured data modalities, such as text or single cell sequencing data, where multiple data points are measured per each sample and exhibit a clustering structure. To overcome this challenge, we introduce FACTM, a novel, multi-view and multi-structure Bayesian model that combines FA with correlated topic modeling and is optimized using variational inference. Additionally, we introduce a method for rotating latent factors to enhance interpretability with respect to binary features. On text and video benchmarks as well as real-world music and COVID-19 datasets, we demonstrate that FACTM outperforms other methods in identifying clusters in structured data, and integrating them with simple modalities via the inference of shared, interpretable factors.} }
Endnote
%0 Conference Paper %T Factor Analysis with Correlated Topic Model for Multi-Modal Data %A Małgorzata Łazęcka %A Ewa Maria Szczurek %B Proceedings of The 28th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2025 %E Yingzhen Li %E Stephan Mandt %E Shipra Agrawal %E Emtiyaz Khan %F pmlr-v258-lazecka25a %I PMLR %P 1801--1809 %U https://proceedings.mlr.press/v258/lazecka25a.html %V 258 %X Integrating various data modalities brings valuable insights into underlying phenomena. Multimodal factor analysis (FA) uncovers shared axes of variation underlying different simple data modalities, where each sample is represented by a vector of features. However, FA is not suited for structured data modalities, such as text or single cell sequencing data, where multiple data points are measured per each sample and exhibit a clustering structure. To overcome this challenge, we introduce FACTM, a novel, multi-view and multi-structure Bayesian model that combines FA with correlated topic modeling and is optimized using variational inference. Additionally, we introduce a method for rotating latent factors to enhance interpretability with respect to binary features. On text and video benchmarks as well as real-world music and COVID-19 datasets, we demonstrate that FACTM outperforms other methods in identifying clusters in structured data, and integrating them with simple modalities via the inference of shared, interpretable factors.
APA
Łazęcka, M. & Szczurek, E.M.. (2025). Factor Analysis with Correlated Topic Model for Multi-Modal Data. Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 258:1801-1809 Available from https://proceedings.mlr.press/v258/lazecka25a.html.

Related Material