From Feature Interaction to Feature Generation: A Generative Paradigm of CTR Prediction Models

Mingjia Yin, Junwei Pan, Hao Wang, Ximei Wang, Shangyu Zhang, Jie Jiang, Defu Lian, Enhong Chen
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:72393-72413, 2025.

Abstract

Click-Through Rate (CTR) prediction, a core task in recommendation systems, aims to estimate the probability of users clicking on items. Existing models predominantly follow a discriminative paradigm, which relies heavily on explicit interactions between raw ID embeddings. However, this paradigm inherently renders them susceptible to two critical issues: embedding dimensional collapse and information redundancy, stemming from the over-reliance on feature interactions over raw ID embeddings. To address these limitations, we propose a novel Supervised Feature Generation (SFG) framework, shifting the paradigm from discriminative "feature interaction" to generative "feature generation". Specifically, SFG comprises two key components: an Encoder that constructs hidden embeddings for each feature, and a Decoder tasked with regenerating the feature embeddings of all features from these hidden representations. Unlike existing generative approaches that adopt self-supervised losses, we introduce a supervised loss to utilize the supervised signal, i.e., click or not, in the CTR prediction task. This framework exhibits strong generalizability: it can be seamlessly integrated with most existing CTR models, reformulating them under the generative paradigm. Extensive experiments demonstrate that SFG consistently mitigates embedding collapse and reduces information redundancy, while yielding substantial performance gains across various datasets and base models. The code is available at https://github.com/USTC-StarTeam/GE4Rec.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-yin25c, title = {From Feature Interaction to Feature Generation: A Generative Paradigm of {CTR} Prediction Models}, author = {Yin, Mingjia and Pan, Junwei and Wang, Hao and Wang, Ximei and Zhang, Shangyu and Jiang, Jie and Lian, Defu and Chen, Enhong}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {72393--72413}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/yin25c/yin25c.pdf}, url = {https://proceedings.mlr.press/v267/yin25c.html}, abstract = {Click-Through Rate (CTR) prediction, a core task in recommendation systems, aims to estimate the probability of users clicking on items. Existing models predominantly follow a discriminative paradigm, which relies heavily on explicit interactions between raw ID embeddings. However, this paradigm inherently renders them susceptible to two critical issues: embedding dimensional collapse and information redundancy, stemming from the over-reliance on feature interactions over raw ID embeddings. To address these limitations, we propose a novel Supervised Feature Generation (SFG) framework, shifting the paradigm from discriminative "feature interaction" to generative "feature generation". Specifically, SFG comprises two key components: an Encoder that constructs hidden embeddings for each feature, and a Decoder tasked with regenerating the feature embeddings of all features from these hidden representations. Unlike existing generative approaches that adopt self-supervised losses, we introduce a supervised loss to utilize the supervised signal, i.e., click or not, in the CTR prediction task. This framework exhibits strong generalizability: it can be seamlessly integrated with most existing CTR models, reformulating them under the generative paradigm. Extensive experiments demonstrate that SFG consistently mitigates embedding collapse and reduces information redundancy, while yielding substantial performance gains across various datasets and base models. The code is available at https://github.com/USTC-StarTeam/GE4Rec.} }
Endnote
%0 Conference Paper %T From Feature Interaction to Feature Generation: A Generative Paradigm of CTR Prediction Models %A Mingjia Yin %A Junwei Pan %A Hao Wang %A Ximei Wang %A Shangyu Zhang %A Jie Jiang %A Defu Lian %A Enhong Chen %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-yin25c %I PMLR %P 72393--72413 %U https://proceedings.mlr.press/v267/yin25c.html %V 267 %X Click-Through Rate (CTR) prediction, a core task in recommendation systems, aims to estimate the probability of users clicking on items. Existing models predominantly follow a discriminative paradigm, which relies heavily on explicit interactions between raw ID embeddings. However, this paradigm inherently renders them susceptible to two critical issues: embedding dimensional collapse and information redundancy, stemming from the over-reliance on feature interactions over raw ID embeddings. To address these limitations, we propose a novel Supervised Feature Generation (SFG) framework, shifting the paradigm from discriminative "feature interaction" to generative "feature generation". Specifically, SFG comprises two key components: an Encoder that constructs hidden embeddings for each feature, and a Decoder tasked with regenerating the feature embeddings of all features from these hidden representations. Unlike existing generative approaches that adopt self-supervised losses, we introduce a supervised loss to utilize the supervised signal, i.e., click or not, in the CTR prediction task. This framework exhibits strong generalizability: it can be seamlessly integrated with most existing CTR models, reformulating them under the generative paradigm. Extensive experiments demonstrate that SFG consistently mitigates embedding collapse and reduces information redundancy, while yielding substantial performance gains across various datasets and base models. The code is available at https://github.com/USTC-StarTeam/GE4Rec.
APA
Yin, M., Pan, J., Wang, H., Wang, X., Zhang, S., Jiang, J., Lian, D. & Chen, E.. (2025). From Feature Interaction to Feature Generation: A Generative Paradigm of CTR Prediction Models. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:72393-72413 Available from https://proceedings.mlr.press/v267/yin25c.html.

Related Material