Copyright-Bench: Agentic Evaluation of Copyright Law Compliance

Zheng Hui, Doni Bloomfield, Noam Kolt
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:48774-48793, 2026.

Abstract

Large language model (LLM) agents increasingly perform commercial tasks that involve retrieving external content such as images and, where appropriate, reproducing that content. LLM agents should comply with the law, including copyright law. Presently, however, we lack adequate frameworks to assess whether they do so in practice. To that end, we introduce Copyright-Bench, a benchmark designed to evaluate LLM agents’ compliance with copyright law. Copyright-Bench is comprised of realistic commercial tasks—website development, merchandise design, and pitch deck production—that involve agents selecting between public-domain content (the use of which is legal) and copyrighted content (the use of which is infringing in this setting). The evaluation introduces prompt variations that simulate different user preferences, as well as time pressure. Comparing state-of-the-art LLM agents against a human baseline, we find that: (1) agents select copyrighted works despite the availability of public-domain alternatives; and (2) for open-weights models, violation rates increase in response to certain user preferences and simulated time pressure.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-hui26a, title = {Copyright-Bench: Agentic Evaluation of Copyright Law Compliance}, author = {Hui, Zheng and Bloomfield, Doni and Kolt, Noam}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {48774--48793}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/hui26a/hui26a.pdf}, url = {https://proceedings.mlr.press/v306/hui26a.html}, abstract = {Large language model (LLM) agents increasingly perform commercial tasks that involve retrieving external content such as images and, where appropriate, reproducing that content. LLM agents should comply with the law, including copyright law. Presently, however, we lack adequate frameworks to assess whether they do so in practice. To that end, we introduce Copyright-Bench, a benchmark designed to evaluate LLM agents’ compliance with copyright law. Copyright-Bench is comprised of realistic commercial tasks—website development, merchandise design, and pitch deck production—that involve agents selecting between public-domain content (the use of which is legal) and copyrighted content (the use of which is infringing in this setting). The evaluation introduces prompt variations that simulate different user preferences, as well as time pressure. Comparing state-of-the-art LLM agents against a human baseline, we find that: (1) agents select copyrighted works despite the availability of public-domain alternatives; and (2) for open-weights models, violation rates increase in response to certain user preferences and simulated time pressure.} }
Endnote
%0 Conference Paper %T Copyright-Bench: Agentic Evaluation of Copyright Law Compliance %A Zheng Hui %A Doni Bloomfield %A Noam Kolt %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-hui26a %I PMLR %P 48774--48793 %U https://proceedings.mlr.press/v306/hui26a.html %V 306 %X Large language model (LLM) agents increasingly perform commercial tasks that involve retrieving external content such as images and, where appropriate, reproducing that content. LLM agents should comply with the law, including copyright law. Presently, however, we lack adequate frameworks to assess whether they do so in practice. To that end, we introduce Copyright-Bench, a benchmark designed to evaluate LLM agents’ compliance with copyright law. Copyright-Bench is comprised of realistic commercial tasks—website development, merchandise design, and pitch deck production—that involve agents selecting between public-domain content (the use of which is legal) and copyrighted content (the use of which is infringing in this setting). The evaluation introduces prompt variations that simulate different user preferences, as well as time pressure. Comparing state-of-the-art LLM agents against a human baseline, we find that: (1) agents select copyrighted works despite the availability of public-domain alternatives; and (2) for open-weights models, violation rates increase in response to certain user preferences and simulated time pressure.
APA
Hui, Z., Bloomfield, D. & Kolt, N.. (2026). Copyright-Bench: Agentic Evaluation of Copyright Law Compliance. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:48774-48793 Available from https://proceedings.mlr.press/v306/hui26a.html.

Related Material