An Information-Theoretic Criterion for Efficient Data Synthesis

Hanyu Li, Zhengqi Sun, Xiaotie Deng
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:70064-70075, 2026.

Abstract

Synthetic data becomes crucial for large language model training, but its effectiveness is highly inconsistent. We provide an information-theoretic account of this inconsistency: synthetic data improves a model only when the generation-training loop is information-open, i.e., shaped by external signals (verifiers, environments, or rubrics) that inject task-relevant information beyond the model’s current distribution. When the loop is information-closed (relying on the model’s own outputs without such signals), the data processing inequality ensures that task-relevant information can only decrease, making collapse a predicted outcome. Among information-open pipelines, both efficiency and generalization hinge on the meta-level of supervision: a coarser signal such as binary correctness treats all acceptable outputs as equivalent, so the behavior it teaches is not tied to any particular domain or surface form and generalizes naturally across tasks and domains. These observations lead to a guiding thesis: learning preferentially converges to the most information-efficient signal component available, which accelerates learning when that component is the intended one, but causes reward hacking when a spurious pattern happens to be simpler.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-li26fp, title = {An Information-Theoretic Criterion for Efficient Data Synthesis}, author = {Li, Hanyu and Sun, Zhengqi and Deng, Xiaotie}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {70064--70075}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/li26fp/li26fp.pdf}, url = {https://proceedings.mlr.press/v306/li26fp.html}, abstract = {Synthetic data becomes crucial for large language model training, but its effectiveness is highly inconsistent. We provide an information-theoretic account of this inconsistency: synthetic data improves a model only when the generation-training loop is information-open, i.e., shaped by external signals (verifiers, environments, or rubrics) that inject task-relevant information beyond the model’s current distribution. When the loop is information-closed (relying on the model’s own outputs without such signals), the data processing inequality ensures that task-relevant information can only decrease, making collapse a predicted outcome. Among information-open pipelines, both efficiency and generalization hinge on the meta-level of supervision: a coarser signal such as binary correctness treats all acceptable outputs as equivalent, so the behavior it teaches is not tied to any particular domain or surface form and generalizes naturally across tasks and domains. These observations lead to a guiding thesis: learning preferentially converges to the most information-efficient signal component available, which accelerates learning when that component is the intended one, but causes reward hacking when a spurious pattern happens to be simpler.} }
Endnote
%0 Conference Paper %T An Information-Theoretic Criterion for Efficient Data Synthesis %A Hanyu Li %A Zhengqi Sun %A Xiaotie Deng %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-li26fp %I PMLR %P 70064--70075 %U https://proceedings.mlr.press/v306/li26fp.html %V 306 %X Synthetic data becomes crucial for large language model training, but its effectiveness is highly inconsistent. We provide an information-theoretic account of this inconsistency: synthetic data improves a model only when the generation-training loop is information-open, i.e., shaped by external signals (verifiers, environments, or rubrics) that inject task-relevant information beyond the model’s current distribution. When the loop is information-closed (relying on the model’s own outputs without such signals), the data processing inequality ensures that task-relevant information can only decrease, making collapse a predicted outcome. Among information-open pipelines, both efficiency and generalization hinge on the meta-level of supervision: a coarser signal such as binary correctness treats all acceptable outputs as equivalent, so the behavior it teaches is not tied to any particular domain or surface form and generalizes naturally across tasks and domains. These observations lead to a guiding thesis: learning preferentially converges to the most information-efficient signal component available, which accelerates learning when that component is the intended one, but causes reward hacking when a spurious pattern happens to be simpler.
APA
Li, H., Sun, Z. & Deng, X.. (2026). An Information-Theoretic Criterion for Efficient Data Synthesis. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:70064-70075 Available from https://proceedings.mlr.press/v306/li26fp.html.

Related Material