Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMs

Mohammad Jalali, Azim Ospanov, Amin Gohari, Farzan Farnia
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:1369-1377, 2026.

Abstract

Generative models guided by text prompts are widely evaluated for fidelity and prompt alignment, yet their ability to produce diverse outputs remains underexplored. Existing diversity metrics such as Vendi and RKE, which are based on the von Neumann and R{é}nyi entropies of kernel matrices, were developed for unconditional models and cannot distinguish prompt-induced from model-induced variability. We address this gap by introducing \emph{Conditional-Vendi} and \emph{Conditional-RKE}, diversity measures derived from the conditional entropy of positive semidefinite matrices. These scores isolate model-induced diversity in prompt-guided generation, with Conditional-RKE enjoying an $O(1/\sqrt{n})$ convergence rate. For Conditional-Vendi, we introduce a truncated-spectrum approximation that yields scalable and consistent estimates. Experiments on text-to-image, image-captioning, and language generation tasks demonstrate that the conditional scores recover ground-truth diversity orderings and can also guide diffusion models toward more diverse generations. The codebase is available at \url{https://github.com/mjalali/conditional-vendi.}

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-jalali26a, title = { Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMs }, author = {Jalali, Mohammad and Ospanov, Azim and Gohari, Amin and Farnia, Farzan}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {1369--1377}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/jalali26a/jalali26a.pdf}, url = {https://proceedings.mlr.press/v300/jalali26a.html}, abstract = { Generative models guided by text prompts are widely evaluated for fidelity and prompt alignment, yet their ability to produce diverse outputs remains underexplored. Existing diversity metrics such as Vendi and RKE, which are based on the von Neumann and R{é}nyi entropies of kernel matrices, were developed for unconditional models and cannot distinguish prompt-induced from model-induced variability. We address this gap by introducing \emph{Conditional-Vendi} and \emph{Conditional-RKE}, diversity measures derived from the conditional entropy of positive semidefinite matrices. These scores isolate model-induced diversity in prompt-guided generation, with Conditional-RKE enjoying an $O(1/\sqrt{n})$ convergence rate. For Conditional-Vendi, we introduce a truncated-spectrum approximation that yields scalable and consistent estimates. Experiments on text-to-image, image-captioning, and language generation tasks demonstrate that the conditional scores recover ground-truth diversity orderings and can also guide diffusion models toward more diverse generations. The codebase is available at \url{https://github.com/mjalali/conditional-vendi.} } }
Endnote
%0 Conference Paper %T Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMs %A Mohammad Jalali %A Azim Ospanov %A Amin Gohari %A Farzan Farnia %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-jalali26a %I PMLR %P 1369--1377 %U https://proceedings.mlr.press/v300/jalali26a.html %V 300 %X Generative models guided by text prompts are widely evaluated for fidelity and prompt alignment, yet their ability to produce diverse outputs remains underexplored. Existing diversity metrics such as Vendi and RKE, which are based on the von Neumann and R{é}nyi entropies of kernel matrices, were developed for unconditional models and cannot distinguish prompt-induced from model-induced variability. We address this gap by introducing \emph{Conditional-Vendi} and \emph{Conditional-RKE}, diversity measures derived from the conditional entropy of positive semidefinite matrices. These scores isolate model-induced diversity in prompt-guided generation, with Conditional-RKE enjoying an $O(1/\sqrt{n})$ convergence rate. For Conditional-Vendi, we introduce a truncated-spectrum approximation that yields scalable and consistent estimates. Experiments on text-to-image, image-captioning, and language generation tasks demonstrate that the conditional scores recover ground-truth diversity orderings and can also guide diffusion models toward more diverse generations. The codebase is available at \url{https://github.com/mjalali/conditional-vendi.}
APA
Jalali, M., Ospanov, A., Gohari, A. & Farnia, F.. (2026). Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMs . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:1369-1377 Available from https://proceedings.mlr.press/v300/jalali26a.html.

Related Material