Probability-Entropy Calibration: An Elastic Indicator for Adaptive Fine-tuning

Wenhao Yu, Shaohang Wei, Jiahong Liu, Yifan Li, Minda Hu, Aiwei Liu, Hao Zhang, Irwin King
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:151307-151333, 2026.

Abstract

Token-level reweighting is a simple yet effective mechanism for controlling supervised fine-tuning, but common indicators are largely one-dimensional: the ground-truth probability reflects downstream alignment, while token entropy reflects intrinsic uncertainty induced by the pre-training prior. Ignoring entropy can misidentify noisy or easily replaceable tokens as learning-critical, while ignoring probability fails to reflect target-specific alignment. RankTuner introduces a probability–entropy calibration signal, the Relative Rank Indicator, by comparing the rank of the ground-truth token with its expected rank under the predictiondiction distribution. The inverse indicator is used as a token-wise Relative Scale to reweight the fine-tuning objective, focusing updates on truly under-learned tokens without over-penalizing intrinsically uncertain positions. Experiments on multiple backbones show consistent improvements on mathematical reasoning benchmarks, transfer gains on out-of-distribution reasoning, and pre code generation performance over probability- or entropy-only reweighting baselines.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-yu26bc, title = {Probability-Entropy Calibration: An Elastic Indicator for Adaptive Fine-tuning}, author = {Yu, Wenhao and Wei, Shaohang and Liu, Jiahong and Li, Yifan and Hu, Minda and Liu, Aiwei and Zhang, Hao and King, Irwin}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {151307--151333}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/yu26bc/yu26bc.pdf}, url = {https://proceedings.mlr.press/v306/yu26bc.html}, abstract = {Token-level reweighting is a simple yet effective mechanism for controlling supervised fine-tuning, but common indicators are largely one-dimensional: the ground-truth probability reflects downstream alignment, while token entropy reflects intrinsic uncertainty induced by the pre-training prior. Ignoring entropy can misidentify noisy or easily replaceable tokens as learning-critical, while ignoring probability fails to reflect target-specific alignment. RankTuner introduces a probability–entropy calibration signal, the Relative Rank Indicator, by comparing the rank of the ground-truth token with its expected rank under the predictiondiction distribution. The inverse indicator is used as a token-wise Relative Scale to reweight the fine-tuning objective, focusing updates on truly under-learned tokens without over-penalizing intrinsically uncertain positions. Experiments on multiple backbones show consistent improvements on mathematical reasoning benchmarks, transfer gains on out-of-distribution reasoning, and pre code generation performance over probability- or entropy-only reweighting baselines.} }
Endnote
%0 Conference Paper %T Probability-Entropy Calibration: An Elastic Indicator for Adaptive Fine-tuning %A Wenhao Yu %A Shaohang Wei %A Jiahong Liu %A Yifan Li %A Minda Hu %A Aiwei Liu %A Hao Zhang %A Irwin King %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-yu26bc %I PMLR %P 151307--151333 %U https://proceedings.mlr.press/v306/yu26bc.html %V 306 %X Token-level reweighting is a simple yet effective mechanism for controlling supervised fine-tuning, but common indicators are largely one-dimensional: the ground-truth probability reflects downstream alignment, while token entropy reflects intrinsic uncertainty induced by the pre-training prior. Ignoring entropy can misidentify noisy or easily replaceable tokens as learning-critical, while ignoring probability fails to reflect target-specific alignment. RankTuner introduces a probability–entropy calibration signal, the Relative Rank Indicator, by comparing the rank of the ground-truth token with its expected rank under the predictiondiction distribution. The inverse indicator is used as a token-wise Relative Scale to reweight the fine-tuning objective, focusing updates on truly under-learned tokens without over-penalizing intrinsically uncertain positions. Experiments on multiple backbones show consistent improvements on mathematical reasoning benchmarks, transfer gains on out-of-distribution reasoning, and pre code generation performance over probability- or entropy-only reweighting baselines.
APA
Yu, W., Wei, S., Liu, J., Li, Y., Hu, M., Liu, A., Zhang, H. & King, I.. (2026). Probability-Entropy Calibration: An Elastic Indicator for Adaptive Fine-tuning. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:151307-151333 Available from https://proceedings.mlr.press/v306/yu26bc.html.

Related Material