A Unifying Framework for Unsupervised Concept Extraction

Chandler Squires
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:4393-4401, 2026.

Abstract

Techniques for \emph{concept extraction}, such as sparse autoencoders and transcoders, aim to extract high-level symbolic concepts from low-level nonsymbolic representations. When these extracted concepts are used for downstream tasks such as model steering and unlearning, it is essential to understand their guarantees, or lack thereof. In this work, we present a unified theoretical framework for unsupervised concept extraction, in which we frame the task of concept extraction as identifying a generative model. We present a general meta-theorem for identifiability, which reduces the problem of establishing identifiability guarantees to the problem of characterizing the intersection of two sets. As we demonstrate on a range of widely-used approaches, this meta-theorem substantially simplifies the task of proving such guarantees, thus paving the way for the development of new, principled approaches for concept extraction.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-squires26a, title = { A Unifying Framework for Unsupervised Concept Extraction }, author = {Squires, Chandler}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {4393--4401}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/squires26a/squires26a.pdf}, url = {https://proceedings.mlr.press/v300/squires26a.html}, abstract = { Techniques for \emph{concept extraction}, such as sparse autoencoders and transcoders, aim to extract high-level symbolic concepts from low-level nonsymbolic representations. When these extracted concepts are used for downstream tasks such as model steering and unlearning, it is essential to understand their guarantees, or lack thereof. In this work, we present a unified theoretical framework for unsupervised concept extraction, in which we frame the task of concept extraction as identifying a generative model. We present a general meta-theorem for identifiability, which reduces the problem of establishing identifiability guarantees to the problem of characterizing the intersection of two sets. As we demonstrate on a range of widely-used approaches, this meta-theorem substantially simplifies the task of proving such guarantees, thus paving the way for the development of new, principled approaches for concept extraction. } }
Endnote
%0 Conference Paper %T A Unifying Framework for Unsupervised Concept Extraction %A Chandler Squires %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-squires26a %I PMLR %P 4393--4401 %U https://proceedings.mlr.press/v300/squires26a.html %V 300 %X Techniques for \emph{concept extraction}, such as sparse autoencoders and transcoders, aim to extract high-level symbolic concepts from low-level nonsymbolic representations. When these extracted concepts are used for downstream tasks such as model steering and unlearning, it is essential to understand their guarantees, or lack thereof. In this work, we present a unified theoretical framework for unsupervised concept extraction, in which we frame the task of concept extraction as identifying a generative model. We present a general meta-theorem for identifiability, which reduces the problem of establishing identifiability guarantees to the problem of characterizing the intersection of two sets. As we demonstrate on a range of widely-used approaches, this meta-theorem substantially simplifies the task of proving such guarantees, thus paving the way for the development of new, principled approaches for concept extraction.
APA
Squires, C.. (2026). A Unifying Framework for Unsupervised Concept Extraction . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:4393-4401 Available from https://proceedings.mlr.press/v300/squires26a.html.

Related Material