Evaluating Role-Based Prompt Architectures in In-Context Learning

Hamidreza Rouzegar, Masoud Makrehchi
Proceedings of the The 39th Canadian Conference on Artificial Intelligence, PMLR 318:1020-1027, 2026.

Abstract

In-context learning (ICL) enables Large Language Models (LLMs) to generate predictions based on prompts without additional fine-tuning. While prompt engineering has been widely studied, the impact of role design within prompts remains underexplored. This study examines the influence of role configurations in zero-shot and few-shot learning scenarios using GPT-3.5 and GPT-4o from OpenAI and Llama2-7b and Llama2-13b from Meta. We evaluate the models’ performance across datasets, focusing on tasks like sentiment analysis, text classification, question answering, and math reasoning. Our findings suggest the potential of role-based prompt structuring to enhance LLM performance.

Cite this Paper

BibTeX
@InProceedings{pmlr-v318-rouzegar26a, title = {Evaluating Role-Based Prompt Architectures in In-Context Learning}, author = {Rouzegar, Hamidreza and Makrehchi, Masoud}, booktitle = {Proceedings of the The 39th Canadian Conference on Artificial Intelligence}, pages = {1020--1027}, year = {2026}, editor = {Bouzar-Benlabiod, Lydia and Leung, Carson}, volume = {318}, series = {Proceedings of Machine Learning Research}, month = {25--29 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v318/main/assets/rouzegar26a/rouzegar26a.pdf}, url = {https://proceedings.mlr.press/v318/rouzegar26a.html}, abstract = {In-context learning (ICL) enables Large Language Models (LLMs) to generate predictions based on prompts without additional fine-tuning. While prompt engineering has been widely studied, the impact of role design within prompts remains underexplored. This study examines the influence of role configurations in zero-shot and few-shot learning scenarios using GPT-3.5 and GPT-4o from OpenAI and Llama2-7b and Llama2-13b from Meta. We evaluate the models’ performance across datasets, focusing on tasks like sentiment analysis, text classification, question answering, and math reasoning. Our findings suggest the potential of role-based prompt structuring to enhance LLM performance.} }
Endnote
%0 Conference Paper %T Evaluating Role-Based Prompt Architectures in In-Context Learning %A Hamidreza Rouzegar %A Masoud Makrehchi %B Proceedings of the The 39th Canadian Conference on Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Lydia Bouzar-Benlabiod %E Carson Leung %F pmlr-v318-rouzegar26a %I PMLR %P 1020--1027 %U https://proceedings.mlr.press/v318/rouzegar26a.html %V 318 %X In-context learning (ICL) enables Large Language Models (LLMs) to generate predictions based on prompts without additional fine-tuning. While prompt engineering has been widely studied, the impact of role design within prompts remains underexplored. This study examines the influence of role configurations in zero-shot and few-shot learning scenarios using GPT-3.5 and GPT-4o from OpenAI and Llama2-7b and Llama2-13b from Meta. We evaluate the models’ performance across datasets, focusing on tasks like sentiment analysis, text classification, question answering, and math reasoning. Our findings suggest the potential of role-based prompt structuring to enhance LLM performance.
APA
Rouzegar, H. & Makrehchi, M.. (2026). Evaluating Role-Based Prompt Architectures in In-Context Learning. Proceedings of the The 39th Canadian Conference on Artificial Intelligence, in Proceedings of Machine Learning Research 318:1020-1027 Available from https://proceedings.mlr.press/v318/rouzegar26a.html.

Related Material