ActionPiece: Contextually Tokenizing Action Sequences for Generative Recommendation

Yupeng Hou, Jianmo Ni, Zhankui He, Noveen Sachdeva, Wang-Cheng Kang, Ed H. Chi, Julian Mcauley, Derek Zhiyuan Cheng
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:24004-24024, 2025.

Abstract

Generative recommendation (GR) is an emerging paradigm where user actions are tokenized into discrete token patterns and autoregressively generated as predictions. However, existing GR models tokenize each action independently, assigning the same fixed tokens to identical actions across all sequences without considering contextual relationships. This lack of context-awareness can lead to suboptimal performance, as the same action may hold different meanings depending on its surrounding context. To address this issue, we propose ActionPiece to explicitly incorporate context when tokenizing action sequences. In ActionPiece, each action is represented as a set of item features. Given the action sequence corpora, we construct the vocabulary by merging feature patterns as new tokens, based on their co-occurrence frequency both within individual sets and across adjacent sets. Considering the unordered nature of feature sets, we further introduce set permutation regularization, which produces multiple segmentations of action sequences with the same semantics. Our code is available at: https://github.com/google-deepmind/action_piece.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-hou25f, title = {{A}ction{P}iece: Contextually Tokenizing Action Sequences for Generative Recommendation}, author = {Hou, Yupeng and Ni, Jianmo and He, Zhankui and Sachdeva, Noveen and Kang, Wang-Cheng and Chi, Ed H. and Mcauley, Julian and Cheng, Derek Zhiyuan}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {24004--24024}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/hou25f/hou25f.pdf}, url = {https://proceedings.mlr.press/v267/hou25f.html}, abstract = {Generative recommendation (GR) is an emerging paradigm where user actions are tokenized into discrete token patterns and autoregressively generated as predictions. However, existing GR models tokenize each action independently, assigning the same fixed tokens to identical actions across all sequences without considering contextual relationships. This lack of context-awareness can lead to suboptimal performance, as the same action may hold different meanings depending on its surrounding context. To address this issue, we propose ActionPiece to explicitly incorporate context when tokenizing action sequences. In ActionPiece, each action is represented as a set of item features. Given the action sequence corpora, we construct the vocabulary by merging feature patterns as new tokens, based on their co-occurrence frequency both within individual sets and across adjacent sets. Considering the unordered nature of feature sets, we further introduce set permutation regularization, which produces multiple segmentations of action sequences with the same semantics. Our code is available at: https://github.com/google-deepmind/action_piece.} }
Endnote
%0 Conference Paper %T ActionPiece: Contextually Tokenizing Action Sequences for Generative Recommendation %A Yupeng Hou %A Jianmo Ni %A Zhankui He %A Noveen Sachdeva %A Wang-Cheng Kang %A Ed H. Chi %A Julian Mcauley %A Derek Zhiyuan Cheng %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-hou25f %I PMLR %P 24004--24024 %U https://proceedings.mlr.press/v267/hou25f.html %V 267 %X Generative recommendation (GR) is an emerging paradigm where user actions are tokenized into discrete token patterns and autoregressively generated as predictions. However, existing GR models tokenize each action independently, assigning the same fixed tokens to identical actions across all sequences without considering contextual relationships. This lack of context-awareness can lead to suboptimal performance, as the same action may hold different meanings depending on its surrounding context. To address this issue, we propose ActionPiece to explicitly incorporate context when tokenizing action sequences. In ActionPiece, each action is represented as a set of item features. Given the action sequence corpora, we construct the vocabulary by merging feature patterns as new tokens, based on their co-occurrence frequency both within individual sets and across adjacent sets. Considering the unordered nature of feature sets, we further introduce set permutation regularization, which produces multiple segmentations of action sequences with the same semantics. Our code is available at: https://github.com/google-deepmind/action_piece.
APA
Hou, Y., Ni, J., He, Z., Sachdeva, N., Kang, W., Chi, E.H., Mcauley, J. & Cheng, D.Z.. (2025). ActionPiece: Contextually Tokenizing Action Sequences for Generative Recommendation. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:24004-24024 Available from https://proceedings.mlr.press/v267/hou25f.html.

Related Material