Learning the RoPEs: Better 2D and 3D Position Encodings with STRING

Connor Schenck, Isaac Reid, Mithun George Jacob, Alex Bewley, Joshua Ainslie, David Rendleman, Deepali Jain, Mohit Sharma, Kumar Avinava Dubey, Ayzaan Wahid, Sumeet Singh, René Wagner, Tianli Ding, Chuyuan Fu, Arunkumar Byravan, Jake Varley, Alexey A. Gritsenko, Matthias Minderer, Dmitry Kalashnikov, Jonathan Tompson, Vikas Sindhwani, Krzysztof Marcin Choromanski
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:53295-53315, 2025.

Abstract

We introduce $\textbf{STRING}$: Separable Translationally Invariant Position Encodings. STRING extends Rotary Position Encodings, a recently proposed and widely used algorithm in large language models, via a unifying theoretical framework. Importantly, STRING still provides $\textbf{exact}$ translation invariance, including token coordinates of arbitrary dimensionality, whilst maintaining a low computational footprint. These properties are especially important in robotics, where efficient 3D token representation is key. We integrate STRING into Vision Transformers with RGB(-D) inputs (color plus optional depth), showing substantial gains, e.g. in open-vocabulary object detection and for robotics controllers. We complement our experiments with a rigorous mathematical analysis, proving the universality of our methods. Videos of STRING-based robotics controllers can be found here: https://sites.google.com/view/string-robotics.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-schenck25a, title = {Learning the {R}o{PE}s: Better 2{D} and 3{D} Position Encodings with {STRING}}, author = {Schenck, Connor and Reid, Isaac and Jacob, Mithun George and Bewley, Alex and Ainslie, Joshua and Rendleman, David and Jain, Deepali and Sharma, Mohit and Dubey, Kumar Avinava and Wahid, Ayzaan and Singh, Sumeet and Wagner, Ren\'{e} and Ding, Tianli and Fu, Chuyuan and Byravan, Arunkumar and Varley, Jake and Gritsenko, Alexey A. and Minderer, Matthias and Kalashnikov, Dmitry and Tompson, Jonathan and Sindhwani, Vikas and Choromanski, Krzysztof Marcin}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {53295--53315}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/schenck25a/schenck25a.pdf}, url = {https://proceedings.mlr.press/v267/schenck25a.html}, abstract = {We introduce $\textbf{STRING}$: Separable Translationally Invariant Position Encodings. STRING extends Rotary Position Encodings, a recently proposed and widely used algorithm in large language models, via a unifying theoretical framework. Importantly, STRING still provides $\textbf{exact}$ translation invariance, including token coordinates of arbitrary dimensionality, whilst maintaining a low computational footprint. These properties are especially important in robotics, where efficient 3D token representation is key. We integrate STRING into Vision Transformers with RGB(-D) inputs (color plus optional depth), showing substantial gains, e.g. in open-vocabulary object detection and for robotics controllers. We complement our experiments with a rigorous mathematical analysis, proving the universality of our methods. Videos of STRING-based robotics controllers can be found here: https://sites.google.com/view/string-robotics.} }
Endnote
%0 Conference Paper %T Learning the RoPEs: Better 2D and 3D Position Encodings with STRING %A Connor Schenck %A Isaac Reid %A Mithun George Jacob %A Alex Bewley %A Joshua Ainslie %A David Rendleman %A Deepali Jain %A Mohit Sharma %A Kumar Avinava Dubey %A Ayzaan Wahid %A Sumeet Singh %A René Wagner %A Tianli Ding %A Chuyuan Fu %A Arunkumar Byravan %A Jake Varley %A Alexey A. Gritsenko %A Matthias Minderer %A Dmitry Kalashnikov %A Jonathan Tompson %A Vikas Sindhwani %A Krzysztof Marcin Choromanski %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-schenck25a %I PMLR %P 53295--53315 %U https://proceedings.mlr.press/v267/schenck25a.html %V 267 %X We introduce $\textbf{STRING}$: Separable Translationally Invariant Position Encodings. STRING extends Rotary Position Encodings, a recently proposed and widely used algorithm in large language models, via a unifying theoretical framework. Importantly, STRING still provides $\textbf{exact}$ translation invariance, including token coordinates of arbitrary dimensionality, whilst maintaining a low computational footprint. These properties are especially important in robotics, where efficient 3D token representation is key. We integrate STRING into Vision Transformers with RGB(-D) inputs (color plus optional depth), showing substantial gains, e.g. in open-vocabulary object detection and for robotics controllers. We complement our experiments with a rigorous mathematical analysis, proving the universality of our methods. Videos of STRING-based robotics controllers can be found here: https://sites.google.com/view/string-robotics.
APA
Schenck, C., Reid, I., Jacob, M.G., Bewley, A., Ainslie, J., Rendleman, D., Jain, D., Sharma, M., Dubey, K.A., Wahid, A., Singh, S., Wagner, R., Ding, T., Fu, C., Byravan, A., Varley, J., Gritsenko, A.A., Minderer, M., Kalashnikov, D., Tompson, J., Sindhwani, V. & Choromanski, K.M.. (2025). Learning the RoPEs: Better 2D and 3D Position Encodings with STRING. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:53295-53315 Available from https://proceedings.mlr.press/v267/schenck25a.html.

Related Material