An Evaluation of Cost Functions for Algorithmic Recourse

Eoin M. Kenny, Allan Anzagira, Tom Bewley, Freddy Lecue, Manuela Veloso
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:2863-2871, 2026.

Abstract

Algorithmic recourse is a field concerned with offering actionable recommendations to individuals who have received adverse outcomes from automated systems. Most recourse algorithms assume access to a cost function, which quantifies the effort involved in following these suggestions. However, to date, there has been no serious benchmarking of these functions both from a computational and human perspective. In this paper, we propose four metrics to evaluate whether currently popular cost functions in recourse satisfy the minimal requirements for meaningful distance calculations. In addition, we also propose extensions to current approaches using large-language models (LLMs) as surrogate human labellers, which are prompted with a cost-based desiderata. Experiments revealed that methods focused on the Bradley-Terry model perform best, but only when scaled up with our proposed LLM extensions, which would be the recommended choice in practice. We expect our insights to help practitioners in training and designing appropriate cost functions in the future.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-kenny26a, title = { An Evaluation of Cost Functions for Algorithmic Recourse }, author = {Kenny, Eoin M. and Anzagira, Allan and Bewley, Tom and Lecue, Freddy and Veloso, Manuela}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {2863--2871}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/kenny26a/kenny26a.pdf}, url = {https://proceedings.mlr.press/v300/kenny26a.html}, abstract = { Algorithmic recourse is a field concerned with offering actionable recommendations to individuals who have received adverse outcomes from automated systems. Most recourse algorithms assume access to a cost function, which quantifies the effort involved in following these suggestions. However, to date, there has been no serious benchmarking of these functions both from a computational and human perspective. In this paper, we propose four metrics to evaluate whether currently popular cost functions in recourse satisfy the minimal requirements for meaningful distance calculations. In addition, we also propose extensions to current approaches using large-language models (LLMs) as surrogate human labellers, which are prompted with a cost-based desiderata. Experiments revealed that methods focused on the Bradley-Terry model perform best, but only when scaled up with our proposed LLM extensions, which would be the recommended choice in practice. We expect our insights to help practitioners in training and designing appropriate cost functions in the future. } }
Endnote
%0 Conference Paper %T An Evaluation of Cost Functions for Algorithmic Recourse %A Eoin M. Kenny %A Allan Anzagira %A Tom Bewley %A Freddy Lecue %A Manuela Veloso %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-kenny26a %I PMLR %P 2863--2871 %U https://proceedings.mlr.press/v300/kenny26a.html %V 300 %X Algorithmic recourse is a field concerned with offering actionable recommendations to individuals who have received adverse outcomes from automated systems. Most recourse algorithms assume access to a cost function, which quantifies the effort involved in following these suggestions. However, to date, there has been no serious benchmarking of these functions both from a computational and human perspective. In this paper, we propose four metrics to evaluate whether currently popular cost functions in recourse satisfy the minimal requirements for meaningful distance calculations. In addition, we also propose extensions to current approaches using large-language models (LLMs) as surrogate human labellers, which are prompted with a cost-based desiderata. Experiments revealed that methods focused on the Bradley-Terry model perform best, but only when scaled up with our proposed LLM extensions, which would be the recommended choice in practice. We expect our insights to help practitioners in training and designing appropriate cost functions in the future.
APA
Kenny, E.M., Anzagira, A., Bewley, T., Lecue, F. & Veloso, M.. (2026). An Evaluation of Cost Functions for Algorithmic Recourse . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:2863-2871 Available from https://proceedings.mlr.press/v300/kenny26a.html.

Related Material