HeteroForge: Heterogeneous Multi-Agent Debate for Curriculum-Aligned STEM Exercise Generation

Nicy Scaria, Silvester John Joseph Kennedy, Deepak Subramani
Proceedings of the Impactful and Responsible AI Systems for Education Workshop, PMLR 339:211-216, 2026.

Abstract

Large language models are increasingly used to generate educational content, but many existing approaches focus on a single question format and rely on human review or same model validation for quality assurance. This limits scalability and can leave systematic evaluator blind spots unaddressed. We present \textit{HeteroForge}, a three pass pipeline for curriculum-aligned STEM exercise generation. Pass 1 uses separate prompts and JSON schemas for different question formats, followed by validation and self-refinement, to produce structurally valid exercises from curriculum metadata. Pass 2 performs heterogeneous multi-agent debate, where three models critique the exercises across accuracy, grade appropriateness, and scaffolded rigor before a judge model renders a structured verdict. Pass 3 recalibrates rejected exercises using judge feedback and a persistent curriculum-specific evaluation summary. The pipeline is modular across question formats, curriculum taxonomies, and model backends, allowing new formats, curricula, or evaluators to be added without changing the overall architecture.

Cite this Paper

BibTeX
@InProceedings{pmlr-v339-scaria26b, title = {HeteroForge: Heterogeneous Multi-Agent Debate for Curriculum-Aligned STEM Exercise Generation}, author = {Scaria, Nicy and Kennedy, Silvester John Joseph and Subramani, Deepak}, booktitle = {Proceedings of the Impactful and Responsible AI Systems for Education Workshop}, pages = {211--216}, year = {2026}, editor = {Basu Mallick, Debshila and Woodhead, Simon and Wang, Zichao and Ananda, Muktha and Burstein, Jill and Murphy, April}, volume = {339}, series = {Proceedings of Machine Learning Research}, month = {28 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v339/main/assets/scaria26b/scaria26b.pdf}, url = {https://proceedings.mlr.press/v339/scaria26b.html}, abstract = {Large language models are increasingly used to generate educational content, but many existing approaches focus on a single question format and rely on human review or same model validation for quality assurance. This limits scalability and can leave systematic evaluator blind spots unaddressed. We present \textit{HeteroForge}, a three pass pipeline for curriculum-aligned STEM exercise generation. Pass 1 uses separate prompts and JSON schemas for different question formats, followed by validation and self-refinement, to produce structurally valid exercises from curriculum metadata. Pass 2 performs heterogeneous multi-agent debate, where three models critique the exercises across accuracy, grade appropriateness, and scaffolded rigor before a judge model renders a structured verdict. Pass 3 recalibrates rejected exercises using judge feedback and a persistent curriculum-specific evaluation summary. The pipeline is modular across question formats, curriculum taxonomies, and model backends, allowing new formats, curricula, or evaluators to be added without changing the overall architecture.} }
Endnote
%0 Conference Paper %T HeteroForge: Heterogeneous Multi-Agent Debate for Curriculum-Aligned STEM Exercise Generation %A Nicy Scaria %A Silvester John Joseph Kennedy %A Deepak Subramani %B Proceedings of the Impactful and Responsible AI Systems for Education Workshop %C Proceedings of Machine Learning Research %D 2026 %E Debshila Basu Mallick %E Simon Woodhead %E Zichao Wang %E Muktha Ananda %E Jill Burstein %E April Murphy %F pmlr-v339-scaria26b %I PMLR %P 211--216 %U https://proceedings.mlr.press/v339/scaria26b.html %V 339 %X Large language models are increasingly used to generate educational content, but many existing approaches focus on a single question format and rely on human review or same model validation for quality assurance. This limits scalability and can leave systematic evaluator blind spots unaddressed. We present \textit{HeteroForge}, a three pass pipeline for curriculum-aligned STEM exercise generation. Pass 1 uses separate prompts and JSON schemas for different question formats, followed by validation and self-refinement, to produce structurally valid exercises from curriculum metadata. Pass 2 performs heterogeneous multi-agent debate, where three models critique the exercises across accuracy, grade appropriateness, and scaffolded rigor before a judge model renders a structured verdict. Pass 3 recalibrates rejected exercises using judge feedback and a persistent curriculum-specific evaluation summary. The pipeline is modular across question formats, curriculum taxonomies, and model backends, allowing new formats, curricula, or evaluators to be added without changing the overall architecture.
APA
Scaria, N., Kennedy, S.J.J. & Subramani, D.. (2026). HeteroForge: Heterogeneous Multi-Agent Debate for Curriculum-Aligned STEM Exercise Generation. Proceedings of the Impactful and Responsible AI Systems for Education Workshop, in Proceedings of Machine Learning Research 339:211-216 Available from https://proceedings.mlr.press/v339/scaria26b.html.

Related Material