A unified probabilistic model for learning latent factors and their connectivities from high-dimensional data

Connectivity estimation is challenging in the context of high-dimensional data. A useful preprocessing step is to group variables into clusters, however, it is not always clear how to do so from the perspective of connectiv- ity estimation. Another practical challenge is that we may have data from multiple related classes (e.g., multiple subjects or conditions) and wish to incorporate constraints on the simi- larities across classes. We propose a probabilis- tic model which simultaneously performs both a grouping of variables (i.e., detecting commu- nity structure) and estimation of connectivities between the groups which correspond to latent variables. The model is essentially a factor anal- ysis model where the factors are allowed to have arbitrary correlations, while the factor loading matrix is constrained to express a community structure. The model can be applied on multiple classes so that the connectivities can be differ- ent between the classes, while the community structure is the same for all classes. We pro- pose an efficient estimation algorithm based on score matching, and prove the identifiability of the model. Finally, we present an extension to directed (causal) connectivities over latent vari- ables. Simulations and experiments on fMRI data validate the practical utility of the method.