Structured deep generative models for sampling on constraint manifolds in sequential manipulation

Joaquim Ortiz-Haro, Jung-Su Ha, Danny Driess, Marc Toussaint
Proceedings of the 5th Conference on Robot Learning, PMLR 164:213-223, 2022.

Abstract

Sampling efficiently on constraint manifolds is a core problem in robotics. We propose Deep Generative Constraint Sampling (DGCS), which combines a deep generative model for sampling close to a constraint manifold with nonlinear constrained optimization to project to the constraint manifold. The generative model is conditioned on the problem instance, taking a scene image as input, and it is trained with a dataset of solutions and a novel analytic constraint term. To further improve the precision and diversity of samples, we extend the approach to exploit a factorization of the constrained problem. We evaluate our approach in two problems of robotic sequential manipulation in cluttered environments. Experimental results demonstrate that our deep generative model produces diverse and precise samples and outperforms heuristic warmstart initialization.

Cite this Paper

BibTeX
@InProceedings{pmlr-v164-ortiz-haro22a, title = {Structured deep generative models for sampling on constraint manifolds in sequential manipulation}, author = {Ortiz-Haro, Joaquim and Ha, Jung-Su and Driess, Danny and Toussaint, Marc}, booktitle = {Proceedings of the 5th Conference on Robot Learning}, pages = {213--223}, year = {2022}, editor = {Faust, Aleksandra and Hsu, David and Neumann, Gerhard}, volume = {164}, series = {Proceedings of Machine Learning Research}, month = {08--11 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v164/ortiz-haro22a/ortiz-haro22a.pdf}, url = {https://proceedings.mlr.press/v164/ortiz-haro22a.html}, abstract = {Sampling efficiently on constraint manifolds is a core problem in robotics. We propose Deep Generative Constraint Sampling (DGCS), which combines a deep generative model for sampling close to a constraint manifold with nonlinear constrained optimization to project to the constraint manifold. The generative model is conditioned on the problem instance, taking a scene image as input, and it is trained with a dataset of solutions and a novel analytic constraint term. To further improve the precision and diversity of samples, we extend the approach to exploit a factorization of the constrained problem. We evaluate our approach in two problems of robotic sequential manipulation in cluttered environments. Experimental results demonstrate that our deep generative model produces diverse and precise samples and outperforms heuristic warmstart initialization. } }
Endnote
%0 Conference Paper %T Structured deep generative models for sampling on constraint manifolds in sequential manipulation %A Joaquim Ortiz-Haro %A Jung-Su Ha %A Danny Driess %A Marc Toussaint %B Proceedings of the 5th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2022 %E Aleksandra Faust %E David Hsu %E Gerhard Neumann %F pmlr-v164-ortiz-haro22a %I PMLR %P 213--223 %U https://proceedings.mlr.press/v164/ortiz-haro22a.html %V 164 %X Sampling efficiently on constraint manifolds is a core problem in robotics. We propose Deep Generative Constraint Sampling (DGCS), which combines a deep generative model for sampling close to a constraint manifold with nonlinear constrained optimization to project to the constraint manifold. The generative model is conditioned on the problem instance, taking a scene image as input, and it is trained with a dataset of solutions and a novel analytic constraint term. To further improve the precision and diversity of samples, we extend the approach to exploit a factorization of the constrained problem. We evaluate our approach in two problems of robotic sequential manipulation in cluttered environments. Experimental results demonstrate that our deep generative model produces diverse and precise samples and outperforms heuristic warmstart initialization.
APA
Ortiz-Haro, J., Ha, J., Driess, D. & Toussaint, M.. (2022). Structured deep generative models for sampling on constraint manifolds in sequential manipulation. Proceedings of the 5th Conference on Robot Learning, in Proceedings of Machine Learning Research 164:213-223 Available from https://proceedings.mlr.press/v164/ortiz-haro22a.html.

Related Material