CRAG: Can 3D Generative Models Help 3D Assembly?

Zeyu Jiang, Sihang Li, Siqi Tan, Chenyang Xu, Juexiao Zhang, Julia Galway-Witham, Xue Wang, Scott A. Williams, Radu Iovita, Chen Feng, Jing Zhang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:52689-52703, 2026.

Abstract

Most existing 3D assembly methods treat the problem as pure pose estimation, rearranging observed parts via rigid transformations. In contrast, human assembly naturally couples structural reasoning with holistic shape inference. Inspired by this intuition, we reformulate 3D assembly as a joint problem of assembly and generation. We show that these two processes are mutually reinforcing: assembly provides part-level structural priors for generation, while generation injects holistic shape context that resolves ambiguities in assembly. Unlike prior methods that cannot synthesize missing geometry, we propose CRAG, which simultaneously generates plausible complete shapes and predicts poses for input parts. Extensive experiments demonstrate state-of-the-art performance across in-the-wild objects with diverse geometries, varying part counts, and missing pieces. Our code and models will be released.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-jiang26ab, title = {{CRAG}: Can 3{D} Generative Models Help 3{D} Assembly?}, author = {Jiang, Zeyu and Li, Sihang and Tan, Siqi and Xu, Chenyang and Zhang, Juexiao and Galway-Witham, Julia and Wang, Xue and Williams, Scott A. and Iovita, Radu and Feng, Chen and Zhang, Jing}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {52689--52703}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/jiang26ab/jiang26ab.pdf}, url = {https://proceedings.mlr.press/v306/jiang26ab.html}, abstract = {Most existing 3D assembly methods treat the problem as pure pose estimation, rearranging observed parts via rigid transformations. In contrast, human assembly naturally couples structural reasoning with holistic shape inference. Inspired by this intuition, we reformulate 3D assembly as a joint problem of assembly and generation. We show that these two processes are mutually reinforcing: assembly provides part-level structural priors for generation, while generation injects holistic shape context that resolves ambiguities in assembly. Unlike prior methods that cannot synthesize missing geometry, we propose CRAG, which simultaneously generates plausible complete shapes and predicts poses for input parts. Extensive experiments demonstrate state-of-the-art performance across in-the-wild objects with diverse geometries, varying part counts, and missing pieces. Our code and models will be released.} }
Endnote
%0 Conference Paper %T CRAG: Can 3D Generative Models Help 3D Assembly? %A Zeyu Jiang %A Sihang Li %A Siqi Tan %A Chenyang Xu %A Juexiao Zhang %A Julia Galway-Witham %A Xue Wang %A Scott A. Williams %A Radu Iovita %A Chen Feng %A Jing Zhang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-jiang26ab %I PMLR %P 52689--52703 %U https://proceedings.mlr.press/v306/jiang26ab.html %V 306 %X Most existing 3D assembly methods treat the problem as pure pose estimation, rearranging observed parts via rigid transformations. In contrast, human assembly naturally couples structural reasoning with holistic shape inference. Inspired by this intuition, we reformulate 3D assembly as a joint problem of assembly and generation. We show that these two processes are mutually reinforcing: assembly provides part-level structural priors for generation, while generation injects holistic shape context that resolves ambiguities in assembly. Unlike prior methods that cannot synthesize missing geometry, we propose CRAG, which simultaneously generates plausible complete shapes and predicts poses for input parts. Extensive experiments demonstrate state-of-the-art performance across in-the-wild objects with diverse geometries, varying part counts, and missing pieces. Our code and models will be released.
APA
Jiang, Z., Li, S., Tan, S., Xu, C., Zhang, J., Galway-Witham, J., Wang, X., Williams, S.A., Iovita, R., Feng, C. & Zhang, J.. (2026). CRAG: Can 3D Generative Models Help 3D Assembly?. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:52689-52703 Available from https://proceedings.mlr.press/v306/jiang26ab.html.

Related Material