SafeSci: Safety Evaluation of Large Language Models in Science Domains and Beyond

Xiangyang Zhu, Yuan Tian, Qi Jia, Kaiwei Zhang, Zicheng Zhang, Chunyi Li, Zijian Chen, Lu Sun, Kaiyuan Ji, Renrui Zhang, Wei Sun, Guangtao Zhai
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:167724-167752, 2026.

Abstract

The success of large language models (LLMs) in scientific domains has heightened safety concerns, prompting numerous benchmarks to evaluate their scientific safety. Existing benchmarks often suffer from limited risk coverage and a reliance on subjective evaluation. To address these problems, we introduce SafeSci, a comprehensive framework for safety evaluation and enhancement in scientific contexts. SafeSci comprises SafeSciBench, a multi-disciplinary benchmark with 0.25M samples, and SafeSciTrain, a large-scale dataset containing 1.5M samples for safety enhancement. SafeSciBench distinguishes between safety knowledge and risk to cover extensive scopes and employs objective metrics such as deterministically answerable questions to mitigate evaluation bias. We evaluate 24 advanced LLMs and reveal critical vulnerabilities. We also observe that LLMs exhibit varying degrees of excessive refusal behaviors on safety-related issues. For safety enhancement, we demonstrate that fine-tuning on SafeSciTrain significantly enhances the safety alignment of LLMs. Finally, we argue that determining the safety of a scientific question should depend on the specific context, rather than universally categorizing it as safe or unsafe.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-zhu26ar, title = {{S}afe{S}ci: Safety Evaluation of Large Language Models in Science Domains and Beyond}, author = {Zhu, Xiangyang and Tian, Yuan and Jia, Qi and Zhang, Kaiwei and Zhang, Zicheng and Li, Chunyi and Chen, Zijian and Sun, Lu and Ji, Kaiyuan and Zhang, Renrui and Sun, Wei and Zhai, Guangtao}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {167724--167752}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/zhu26ar/zhu26ar.pdf}, url = {https://proceedings.mlr.press/v306/zhu26ar.html}, abstract = {The success of large language models (LLMs) in scientific domains has heightened safety concerns, prompting numerous benchmarks to evaluate their scientific safety. Existing benchmarks often suffer from limited risk coverage and a reliance on subjective evaluation. To address these problems, we introduce SafeSci, a comprehensive framework for safety evaluation and enhancement in scientific contexts. SafeSci comprises SafeSciBench, a multi-disciplinary benchmark with 0.25M samples, and SafeSciTrain, a large-scale dataset containing 1.5M samples for safety enhancement. SafeSciBench distinguishes between safety knowledge and risk to cover extensive scopes and employs objective metrics such as deterministically answerable questions to mitigate evaluation bias. We evaluate 24 advanced LLMs and reveal critical vulnerabilities. We also observe that LLMs exhibit varying degrees of excessive refusal behaviors on safety-related issues. For safety enhancement, we demonstrate that fine-tuning on SafeSciTrain significantly enhances the safety alignment of LLMs. Finally, we argue that determining the safety of a scientific question should depend on the specific context, rather than universally categorizing it as safe or unsafe.} }
Endnote
%0 Conference Paper %T SafeSci: Safety Evaluation of Large Language Models in Science Domains and Beyond %A Xiangyang Zhu %A Yuan Tian %A Qi Jia %A Kaiwei Zhang %A Zicheng Zhang %A Chunyi Li %A Zijian Chen %A Lu Sun %A Kaiyuan Ji %A Renrui Zhang %A Wei Sun %A Guangtao Zhai %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-zhu26ar %I PMLR %P 167724--167752 %U https://proceedings.mlr.press/v306/zhu26ar.html %V 306 %X The success of large language models (LLMs) in scientific domains has heightened safety concerns, prompting numerous benchmarks to evaluate their scientific safety. Existing benchmarks often suffer from limited risk coverage and a reliance on subjective evaluation. To address these problems, we introduce SafeSci, a comprehensive framework for safety evaluation and enhancement in scientific contexts. SafeSci comprises SafeSciBench, a multi-disciplinary benchmark with 0.25M samples, and SafeSciTrain, a large-scale dataset containing 1.5M samples for safety enhancement. SafeSciBench distinguishes between safety knowledge and risk to cover extensive scopes and employs objective metrics such as deterministically answerable questions to mitigate evaluation bias. We evaluate 24 advanced LLMs and reveal critical vulnerabilities. We also observe that LLMs exhibit varying degrees of excessive refusal behaviors on safety-related issues. For safety enhancement, we demonstrate that fine-tuning on SafeSciTrain significantly enhances the safety alignment of LLMs. Finally, we argue that determining the safety of a scientific question should depend on the specific context, rather than universally categorizing it as safe or unsafe.
APA
Zhu, X., Tian, Y., Jia, Q., Zhang, K., Zhang, Z., Li, C., Chen, Z., Sun, L., Ji, K., Zhang, R., Sun, W. & Zhai, G.. (2026). SafeSci: Safety Evaluation of Large Language Models in Science Domains and Beyond. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:167724-167752 Available from https://proceedings.mlr.press/v306/zhu26ar.html.

Related Material