Avoid What You Know: Divergent Trajectory Balance for GFlowNets

Pedro Dall’Antonia, Tiago Silva, Daniel Csillag, Salem Lahlou, Diego Mesquita
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:22671-22688, 2026.

Abstract

Generative Flow Networks (GFlowNets) are a flexible family of amortized samplers trained to generate discrete and compositional objects with probability proportional to a reward function. To this end, they learn a policy function over an intractably large state graph by minimizing a stochastic objective over sampled trajectories. However, learning efficiency is constrained by the model’s ability to rapidly explore diverse high-probability regions during training. To mitigate this issue, recent works have focused on incentivizing the exploration of unvisited and valuable states via curiosity-driven search and self-supervised random network distillation, which tend to waste samples on already well-approximated regions of the state space. In this context, we propose Adaptive Complementary Exploration (ACE), a principled algorithm for the effective exploration of novel and high-probability regions when learning GFlowNets. To achieve this, ACE introduces an exploration GFlowNet explicitly trained to search for high-reward states in regions underexplored by the canonical GFlowNet, which learns to sample from the target distribution. Through extensive experiments, we show that ACE consistently and significantly improves upon prior work in terms of approximation accuracy to the target distribution and discovery rate of diverse high-reward states.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-dall-antonia26a, title = {Avoid What You Know: Divergent Trajectory Balance for {GF}low{N}ets}, author = {Dall'Antonia, Pedro and Silva, Tiago and Csillag, Daniel and Lahlou, Salem and Mesquita, Diego}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {22671--22688}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/dall-antonia26a/dall-antonia26a.pdf}, url = {https://proceedings.mlr.press/v306/dall-antonia26a.html}, abstract = {Generative Flow Networks (GFlowNets) are a flexible family of amortized samplers trained to generate discrete and compositional objects with probability proportional to a reward function. To this end, they learn a policy function over an intractably large state graph by minimizing a stochastic objective over sampled trajectories. However, learning efficiency is constrained by the model’s ability to rapidly explore diverse high-probability regions during training. To mitigate this issue, recent works have focused on incentivizing the exploration of unvisited and valuable states via curiosity-driven search and self-supervised random network distillation, which tend to waste samples on already well-approximated regions of the state space. In this context, we propose Adaptive Complementary Exploration (ACE), a principled algorithm for the effective exploration of novel and high-probability regions when learning GFlowNets. To achieve this, ACE introduces an exploration GFlowNet explicitly trained to search for high-reward states in regions underexplored by the canonical GFlowNet, which learns to sample from the target distribution. Through extensive experiments, we show that ACE consistently and significantly improves upon prior work in terms of approximation accuracy to the target distribution and discovery rate of diverse high-reward states.} }
Endnote
%0 Conference Paper %T Avoid What You Know: Divergent Trajectory Balance for GFlowNets %A Pedro Dall’Antonia %A Tiago Silva %A Daniel Csillag %A Salem Lahlou %A Diego Mesquita %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-dall-antonia26a %I PMLR %P 22671--22688 %U https://proceedings.mlr.press/v306/dall-antonia26a.html %V 306 %X Generative Flow Networks (GFlowNets) are a flexible family of amortized samplers trained to generate discrete and compositional objects with probability proportional to a reward function. To this end, they learn a policy function over an intractably large state graph by minimizing a stochastic objective over sampled trajectories. However, learning efficiency is constrained by the model’s ability to rapidly explore diverse high-probability regions during training. To mitigate this issue, recent works have focused on incentivizing the exploration of unvisited and valuable states via curiosity-driven search and self-supervised random network distillation, which tend to waste samples on already well-approximated regions of the state space. In this context, we propose Adaptive Complementary Exploration (ACE), a principled algorithm for the effective exploration of novel and high-probability regions when learning GFlowNets. To achieve this, ACE introduces an exploration GFlowNet explicitly trained to search for high-reward states in regions underexplored by the canonical GFlowNet, which learns to sample from the target distribution. Through extensive experiments, we show that ACE consistently and significantly improves upon prior work in terms of approximation accuracy to the target distribution and discovery rate of diverse high-reward states.
APA
Dall’Antonia, P., Silva, T., Csillag, D., Lahlou, S. & Mesquita, D.. (2026). Avoid What You Know: Divergent Trajectory Balance for GFlowNets. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:22671-22688 Available from https://proceedings.mlr.press/v306/dall-antonia26a.html.

Related Material