PiML: Automated Machine Learning Workflow Optimization using LLM Agents

Abhishek Chopde, Fardeen Pettiwala, Sankar Kirubananth, Sai Kiran Botla, Pachipulusu Ayyappa Kethan
Proceedings of the Fourth International Conference on Automated Machine Learning, PMLR 293:1/1-42, 2025.

Abstract

In this paper, we introduce PiML-Persistent Iterative Machine Learning agentic framework, a novel automated pipeline specifically designed for solving real-world machine learning (ML) tasks such as Kaggle competitions. PiML integrates iterative reasoning, automated code generation, adaptive memory construction, and systematic debugging to tackle complex problems effectively. To rigorously assess our framework, we selected 26 diverse competitions from the MLE-Bench benchmark, ensuring comprehensive representation across various complexity levels, modalities, competition types, and dataset sizes. We quantitatively compared PiML’s performance to AIDE—the best-performing existing baseline from MLE-Bench—across multiple evaluation metrics: Valid Submission rate, Submissions Above Median, Average Percentile Rank, and Medal Achievement Rate. Using the “o3-mini” model, PiML surpassed the baseline in submissions above median (41.0% vs 30.8%), medal attainment rate (29.5% vs 23.1%), and average percentile rank (44.7% vs 38.8%). These results highlight PiML’s flexibility, robustness, and superior performance on practical and complex ML challenges.

Cite this Paper

BibTeX
@InProceedings{pmlr-v293-chopde25a, title = {PiML: Automated Machine Learning Workflow Optimization using LLM Agents}, author = {Chopde, Abhishek and Pettiwala, Fardeen and Kirubananth, Sankar and Botla, Sai Kiran and Kethan, Pachipulusu Ayyappa}, booktitle = {Proceedings of the Fourth International Conference on Automated Machine Learning}, pages = {1/1--42}, year = {2025}, editor = {Akoglu, Leman and Doerr, Carola and van Rijn, Jan N. and Garnett, Roman and Gardner, Jacob R.}, volume = {293}, series = {Proceedings of Machine Learning Research}, month = {08--11 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v293/main/assets/chopde25a/chopde25a.pdf}, url = {https://proceedings.mlr.press/v293/chopde25a.html}, abstract = {In this paper, we introduce PiML-Persistent Iterative Machine Learning agentic framework, a novel automated pipeline specifically designed for solving real-world machine learning (ML) tasks such as Kaggle competitions. PiML integrates iterative reasoning, automated code generation, adaptive memory construction, and systematic debugging to tackle complex problems effectively. To rigorously assess our framework, we selected 26 diverse competitions from the MLE-Bench benchmark, ensuring comprehensive representation across various complexity levels, modalities, competition types, and dataset sizes. We quantitatively compared PiML’s performance to AIDE—the best-performing existing baseline from MLE-Bench—across multiple evaluation metrics: Valid Submission rate, Submissions Above Median, Average Percentile Rank, and Medal Achievement Rate. Using the “o3-mini” model, PiML surpassed the baseline in submissions above median (41.0% vs 30.8%), medal attainment rate (29.5% vs 23.1%), and average percentile rank (44.7% vs 38.8%). These results highlight PiML’s flexibility, robustness, and superior performance on practical and complex ML challenges.} }
Endnote
%0 Conference Paper %T PiML: Automated Machine Learning Workflow Optimization using LLM Agents %A Abhishek Chopde %A Fardeen Pettiwala %A Sankar Kirubananth %A Sai Kiran Botla %A Pachipulusu Ayyappa Kethan %B Proceedings of the Fourth International Conference on Automated Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Leman Akoglu %E Carola Doerr %E Jan N. van Rijn %E Roman Garnett %E Jacob R. Gardner %F pmlr-v293-chopde25a %I PMLR %P 1/1--42 %U https://proceedings.mlr.press/v293/chopde25a.html %V 293 %X In this paper, we introduce PiML-Persistent Iterative Machine Learning agentic framework, a novel automated pipeline specifically designed for solving real-world machine learning (ML) tasks such as Kaggle competitions. PiML integrates iterative reasoning, automated code generation, adaptive memory construction, and systematic debugging to tackle complex problems effectively. To rigorously assess our framework, we selected 26 diverse competitions from the MLE-Bench benchmark, ensuring comprehensive representation across various complexity levels, modalities, competition types, and dataset sizes. We quantitatively compared PiML’s performance to AIDE—the best-performing existing baseline from MLE-Bench—across multiple evaluation metrics: Valid Submission rate, Submissions Above Median, Average Percentile Rank, and Medal Achievement Rate. Using the “o3-mini” model, PiML surpassed the baseline in submissions above median (41.0% vs 30.8%), medal attainment rate (29.5% vs 23.1%), and average percentile rank (44.7% vs 38.8%). These results highlight PiML’s flexibility, robustness, and superior performance on practical and complex ML challenges.
APA
Chopde, A., Pettiwala, F., Kirubananth, S., Botla, S.K. & Kethan, P.A.. (2025). PiML: Automated Machine Learning Workflow Optimization using LLM Agents. Proceedings of the Fourth International Conference on Automated Machine Learning, in Proceedings of Machine Learning Research 293:1/1-42 Available from https://proceedings.mlr.press/v293/chopde25a.html.

Related Material