SMILE: Extended Deep Submodular Function-Based Instruction and In-context Learning Demonstration Selection

Zihan Chen, Chengshuai Shi, Song Wang, Jundong Li, Cong Shen
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:16609-16629, 2026.

Abstract

Prompt optimization is a key way to steer large language models when fine-tuning is impractical. However, instruction optimization (IO) and in-context learning (ICL) demonstration selection are often optimized separately and combined post hoc, implicitly assuming that a "best” instruction and a "best" demonstration set compose well. In practice, their interactions are strong, making such decoupled pipelines brittle. We propose SMILE, an efficient method that jointly selects instructions and demonstrations. Our key observation is that the ICL performance exhibits consistent diminishing returns across diverse instructions. Leveraging this structure, SMILE learns an instruction-conditioned surrogate aligned with LLM feedback and instantiates it as an Extended Deep Submodular Function that captures sample–sample coverage, sample–query relevance, and sample–instruction compatibility. SMILE then performs greedy, query-adaptive selection of the instruction–demonstration pair. Experiments on six datasets and multiple LLM backbones show that SMILE consistently outperforms IO-only, ICL-only, and existing joint baselines, supporting a context engineering view of prompting: jointly optimizing interacting components rather than tuning them in isolation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-chen26dt, title = {{SMILE}: Extended Deep Submodular Function-Based Instruction and In-context Learning Demonstration Selection}, author = {Chen, Zihan and Shi, Chengshuai and Wang, Song and Li, Jundong and Shen, Cong}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {16609--16629}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/chen26dt/chen26dt.pdf}, url = {https://proceedings.mlr.press/v306/chen26dt.html}, abstract = {Prompt optimization is a key way to steer large language models when fine-tuning is impractical. However, instruction optimization (IO) and in-context learning (ICL) demonstration selection are often optimized separately and combined post hoc, implicitly assuming that a "best” instruction and a "best" demonstration set compose well. In practice, their interactions are strong, making such decoupled pipelines brittle. We propose SMILE, an efficient method that jointly selects instructions and demonstrations. Our key observation is that the ICL performance exhibits consistent diminishing returns across diverse instructions. Leveraging this structure, SMILE learns an instruction-conditioned surrogate aligned with LLM feedback and instantiates it as an Extended Deep Submodular Function that captures sample–sample coverage, sample–query relevance, and sample–instruction compatibility. SMILE then performs greedy, query-adaptive selection of the instruction–demonstration pair. Experiments on six datasets and multiple LLM backbones show that SMILE consistently outperforms IO-only, ICL-only, and existing joint baselines, supporting a context engineering view of prompting: jointly optimizing interacting components rather than tuning them in isolation.} }
Endnote
%0 Conference Paper %T SMILE: Extended Deep Submodular Function-Based Instruction and In-context Learning Demonstration Selection %A Zihan Chen %A Chengshuai Shi %A Song Wang %A Jundong Li %A Cong Shen %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-chen26dt %I PMLR %P 16609--16629 %U https://proceedings.mlr.press/v306/chen26dt.html %V 306 %X Prompt optimization is a key way to steer large language models when fine-tuning is impractical. However, instruction optimization (IO) and in-context learning (ICL) demonstration selection are often optimized separately and combined post hoc, implicitly assuming that a "best” instruction and a "best" demonstration set compose well. In practice, their interactions are strong, making such decoupled pipelines brittle. We propose SMILE, an efficient method that jointly selects instructions and demonstrations. Our key observation is that the ICL performance exhibits consistent diminishing returns across diverse instructions. Leveraging this structure, SMILE learns an instruction-conditioned surrogate aligned with LLM feedback and instantiates it as an Extended Deep Submodular Function that captures sample–sample coverage, sample–query relevance, and sample–instruction compatibility. SMILE then performs greedy, query-adaptive selection of the instruction–demonstration pair. Experiments on six datasets and multiple LLM backbones show that SMILE consistently outperforms IO-only, ICL-only, and existing joint baselines, supporting a context engineering view of prompting: jointly optimizing interacting components rather than tuning them in isolation.
APA
Chen, Z., Shi, C., Wang, S., Li, J. & Shen, C.. (2026). SMILE: Extended Deep Submodular Function-Based Instruction and In-context Learning Demonstration Selection. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:16609-16629 Available from https://proceedings.mlr.press/v306/chen26dt.html.

Related Material