CodeChemist: Test-Time Scaling for Low-Resource Code Generation via Functional Knowledge Transfer

Kaixin Wang, Tianlin Li, Xiaoyu Zhang, Aishan Liu, Xianglong Liu, Ziqi Liu, Zhiqiang Zhang, Jun Zhou, Bin Shi
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:127866-127883, 2026.

Abstract

Code Large Language Models (CodeLLMs) have been widely adopted for Natural Language to Programming Language code generation, powering applications with large user bases. Their performance, however, varies sharply across programming languages (PLs) and is particularly suboptimal for low-resource PLs due to data scarcity, limiting their overall usability. In this work, we introduce CodeChemist, a simple yet effective, training-free test-time scaling framework that transfers the model’s functional knowledge from high-resource to low-resource PLs via synthesized test cases, without relying on external models. Specifically, CodeChemist first applies multi-temperature hedged sampling to generate a pool of candidate solutions in the low-resource PL and synthesizes a set of test inputs. It then estimates uncertainty: when uncertainty is low, it selects the output via in-language majority voting; otherwise, it constructs cross-lingual I/O test oracles by executing high-resource reference programs and selects the candidate with the highest pass rate. Extensive experiments demonstrate that CodeChemist significantly outperforms existing test-time scaling methods, improving code generation for both low-resource PLs (e.g., Lua) and complex-syntax PLs (e.g., C++, Java) without retraining.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-wang26dw, title = {{C}ode{C}hemist: Test-Time Scaling for Low-Resource Code Generation via Functional Knowledge Transfer}, author = {Wang, Kaixin and Li, Tianlin and Zhang, Xiaoyu and Liu, Aishan and Liu, Xianglong and Liu, Ziqi and Zhang, Zhiqiang and Zhou, Jun and Shi, Bin}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {127866--127883}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/wang26dw/wang26dw.pdf}, url = {https://proceedings.mlr.press/v306/wang26dw.html}, abstract = {Code Large Language Models (CodeLLMs) have been widely adopted for Natural Language to Programming Language code generation, powering applications with large user bases. Their performance, however, varies sharply across programming languages (PLs) and is particularly suboptimal for low-resource PLs due to data scarcity, limiting their overall usability. In this work, we introduce CodeChemist, a simple yet effective, training-free test-time scaling framework that transfers the model’s functional knowledge from high-resource to low-resource PLs via synthesized test cases, without relying on external models. Specifically, CodeChemist first applies multi-temperature hedged sampling to generate a pool of candidate solutions in the low-resource PL and synthesizes a set of test inputs. It then estimates uncertainty: when uncertainty is low, it selects the output via in-language majority voting; otherwise, it constructs cross-lingual I/O test oracles by executing high-resource reference programs and selects the candidate with the highest pass rate. Extensive experiments demonstrate that CodeChemist significantly outperforms existing test-time scaling methods, improving code generation for both low-resource PLs (e.g., Lua) and complex-syntax PLs (e.g., C++, Java) without retraining.} }
Endnote
%0 Conference Paper %T CodeChemist: Test-Time Scaling for Low-Resource Code Generation via Functional Knowledge Transfer %A Kaixin Wang %A Tianlin Li %A Xiaoyu Zhang %A Aishan Liu %A Xianglong Liu %A Ziqi Liu %A Zhiqiang Zhang %A Jun Zhou %A Bin Shi %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-wang26dw %I PMLR %P 127866--127883 %U https://proceedings.mlr.press/v306/wang26dw.html %V 306 %X Code Large Language Models (CodeLLMs) have been widely adopted for Natural Language to Programming Language code generation, powering applications with large user bases. Their performance, however, varies sharply across programming languages (PLs) and is particularly suboptimal for low-resource PLs due to data scarcity, limiting their overall usability. In this work, we introduce CodeChemist, a simple yet effective, training-free test-time scaling framework that transfers the model’s functional knowledge from high-resource to low-resource PLs via synthesized test cases, without relying on external models. Specifically, CodeChemist first applies multi-temperature hedged sampling to generate a pool of candidate solutions in the low-resource PL and synthesizes a set of test inputs. It then estimates uncertainty: when uncertainty is low, it selects the output via in-language majority voting; otherwise, it constructs cross-lingual I/O test oracles by executing high-resource reference programs and selects the candidate with the highest pass rate. Extensive experiments demonstrate that CodeChemist significantly outperforms existing test-time scaling methods, improving code generation for both low-resource PLs (e.g., Lua) and complex-syntax PLs (e.g., C++, Java) without retraining.
APA
Wang, K., Li, T., Zhang, X., Liu, A., Liu, X., Liu, Z., Zhang, Z., Zhou, J. & Shi, B.. (2026). CodeChemist: Test-Time Scaling for Low-Resource Code Generation via Functional Knowledge Transfer. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:127866-127883 Available from https://proceedings.mlr.press/v306/wang26dw.html.

Related Material