Mitigating Premature Exploitation in Particle-based Monte Carlo for Inference-Time Scaling

Giorgio Giannone, Guangxuan Xu, Nikhil Shivakumar Nayak, Rohan Mahesh Awhad, Shivchander Sudalairaj, Kai Xu, Akash Srivastava
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:34996-35064, 2026.

Abstract

Inference-Time Scaling (ITS) improves language models by allocating more computation at generation time. Particle Filtering (PF) has emerged as a strong ITS method for complex mathematical reasoning tasks, but it is vulnerable when guided by process reward models, which often assign overconfident scores early in the reasoning process. This causes PF to suffer from premature exploitation: it myopically commits to locally promising trajectories, prunes potentially correct hypotheses, and converges to suboptimal solutions. This failure mode, known as particle impoverishment, is especially severe under constrained computational budgets. To address this, we analyze the problem and identify two root causes: a lack of diversity in the particle set due to overconfident resampling and consequent inability to assess the potential of a reasoning path. We introduce Entropic Particle Filtering (ePF), an algorithm that integrates two new techniques to solve these issues. The first technique, Entropic Annealing (EA), directly mitigates particle impoverishment by monitoring search diversity via entropy; when diversity drops, it intervenes by dynamically annealing the resampling distribution to preserve exploration. The second, an enhancement called Look-ahead Modulation (LaM), adds a predictive guide to evaluate a state’s potential based on its successors. By effectively balancing exploration and exploitation, ePF significantly outperforms strong baselines on challenging math benchmarks, achieving up to a 50% relative improvement in task reward.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-giannone26b, title = {Mitigating Premature Exploitation in Particle-based {M}onte {C}arlo for Inference-Time Scaling}, author = {Giannone, Giorgio and Xu, Guangxuan and Nayak, Nikhil Shivakumar and Awhad, Rohan Mahesh and Sudalairaj, Shivchander and Xu, Kai and Srivastava, Akash}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {34996--35064}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/giannone26b/giannone26b.pdf}, url = {https://proceedings.mlr.press/v306/giannone26b.html}, abstract = {Inference-Time Scaling (ITS) improves language models by allocating more computation at generation time. Particle Filtering (PF) has emerged as a strong ITS method for complex mathematical reasoning tasks, but it is vulnerable when guided by process reward models, which often assign overconfident scores early in the reasoning process. This causes PF to suffer from premature exploitation: it myopically commits to locally promising trajectories, prunes potentially correct hypotheses, and converges to suboptimal solutions. This failure mode, known as particle impoverishment, is especially severe under constrained computational budgets. To address this, we analyze the problem and identify two root causes: a lack of diversity in the particle set due to overconfident resampling and consequent inability to assess the potential of a reasoning path. We introduce Entropic Particle Filtering (ePF), an algorithm that integrates two new techniques to solve these issues. The first technique, Entropic Annealing (EA), directly mitigates particle impoverishment by monitoring search diversity via entropy; when diversity drops, it intervenes by dynamically annealing the resampling distribution to preserve exploration. The second, an enhancement called Look-ahead Modulation (LaM), adds a predictive guide to evaluate a state’s potential based on its successors. By effectively balancing exploration and exploitation, ePF significantly outperforms strong baselines on challenging math benchmarks, achieving up to a 50% relative improvement in task reward.} }
Endnote
%0 Conference Paper %T Mitigating Premature Exploitation in Particle-based Monte Carlo for Inference-Time Scaling %A Giorgio Giannone %A Guangxuan Xu %A Nikhil Shivakumar Nayak %A Rohan Mahesh Awhad %A Shivchander Sudalairaj %A Kai Xu %A Akash Srivastava %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-giannone26b %I PMLR %P 34996--35064 %U https://proceedings.mlr.press/v306/giannone26b.html %V 306 %X Inference-Time Scaling (ITS) improves language models by allocating more computation at generation time. Particle Filtering (PF) has emerged as a strong ITS method for complex mathematical reasoning tasks, but it is vulnerable when guided by process reward models, which often assign overconfident scores early in the reasoning process. This causes PF to suffer from premature exploitation: it myopically commits to locally promising trajectories, prunes potentially correct hypotheses, and converges to suboptimal solutions. This failure mode, known as particle impoverishment, is especially severe under constrained computational budgets. To address this, we analyze the problem and identify two root causes: a lack of diversity in the particle set due to overconfident resampling and consequent inability to assess the potential of a reasoning path. We introduce Entropic Particle Filtering (ePF), an algorithm that integrates two new techniques to solve these issues. The first technique, Entropic Annealing (EA), directly mitigates particle impoverishment by monitoring search diversity via entropy; when diversity drops, it intervenes by dynamically annealing the resampling distribution to preserve exploration. The second, an enhancement called Look-ahead Modulation (LaM), adds a predictive guide to evaluate a state’s potential based on its successors. By effectively balancing exploration and exploitation, ePF significantly outperforms strong baselines on challenging math benchmarks, achieving up to a 50% relative improvement in task reward.
APA
Giannone, G., Xu, G., Nayak, N.S., Awhad, R.M., Sudalairaj, S., Xu, K. & Srivastava, A.. (2026). Mitigating Premature Exploitation in Particle-based Monte Carlo for Inference-Time Scaling. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:34996-35064 Available from https://proceedings.mlr.press/v306/giannone26b.html.

Related Material