LogicPrpBank: A Corpus for Logical Implication and Equivalence

Zhexiong Liu, Jing Zhang, Jiaying Lu, Wenjing Ma, Joyce C. Ho
Proceedings of the 2024 AAAI Conference on Artificial Intelligence, PMLR 257:57-65, 2024.

Abstract

Logic reasoning has been critically needed in problem-solving and decision-making. Although Language Models (LMs) have demonstrated capabilities of handling multiple reasoning tasks (e.g., commonsense reasoning), their ability to reason complex mathematical problems, specifically propositional logic, remains largely underexplored. This lack of exploration can be attributed to the limited availability of annotated corpora. Here, we present a well-labeled propositional logic corpus, LogicPrpBank, containing 7093 Propositional Logic Statements (PLSs) across six mathematical subjects, to study a brand-new task of reasoning logical implication and equivalence. We benchmark LogicPrpBank with widely-used LMs to show that our corpus offers a useful resource for this challenging task and there is ample room for model improvement.

Cite this Paper

BibTeX
@InProceedings{pmlr-v257-liu24a, title = {LogicPrpBank: A Corpus for Logical Implication and Equivalence}, author = {Liu, Zhexiong and Zhang, Jing and Lu, Jiaying and Ma, Wenjing and Ho, Joyce C.}, booktitle = {Proceedings of the 2024 AAAI Conference on Artificial Intelligence}, pages = {57--65}, year = {2024}, editor = {Ananda, Muktha and Malick, Debshila Basu and Burstein, Jill and Liu, Lydia T. and Liu, Zitao and Sharpnack, James and Wang, Zichao and Wang, Serena}, volume = {257}, series = {Proceedings of Machine Learning Research}, month = {26--27 Feb}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v257/main/assets/liu24a/liu24a.pdf}, url = {https://proceedings.mlr.press/v257/liu24a.html}, abstract = {Logic reasoning has been critically needed in problem-solving and decision-making. Although Language Models (LMs) have demonstrated capabilities of handling multiple reasoning tasks (e.g., commonsense reasoning), their ability to reason complex mathematical problems, specifically propositional logic, remains largely underexplored. This lack of exploration can be attributed to the limited availability of annotated corpora. Here, we present a well-labeled propositional logic corpus, LogicPrpBank, containing 7093 Propositional Logic Statements (PLSs) across six mathematical subjects, to study a brand-new task of reasoning logical implication and equivalence. We benchmark LogicPrpBank with widely-used LMs to show that our corpus offers a useful resource for this challenging task and there is ample room for model improvement. } }
Endnote
%0 Conference Paper %T LogicPrpBank: A Corpus for Logical Implication and Equivalence %A Zhexiong Liu %A Jing Zhang %A Jiaying Lu %A Wenjing Ma %A Joyce C. Ho %B Proceedings of the 2024 AAAI Conference on Artificial Intelligence %C Proceedings of Machine Learning Research %D 2024 %E Muktha Ananda %E Debshila Basu Malick %E Jill Burstein %E Lydia T. Liu %E Zitao Liu %E James Sharpnack %E Zichao Wang %E Serena Wang %F pmlr-v257-liu24a %I PMLR %P 57--65 %U https://proceedings.mlr.press/v257/liu24a.html %V 257 %X Logic reasoning has been critically needed in problem-solving and decision-making. Although Language Models (LMs) have demonstrated capabilities of handling multiple reasoning tasks (e.g., commonsense reasoning), their ability to reason complex mathematical problems, specifically propositional logic, remains largely underexplored. This lack of exploration can be attributed to the limited availability of annotated corpora. Here, we present a well-labeled propositional logic corpus, LogicPrpBank, containing 7093 Propositional Logic Statements (PLSs) across six mathematical subjects, to study a brand-new task of reasoning logical implication and equivalence. We benchmark LogicPrpBank with widely-used LMs to show that our corpus offers a useful resource for this challenging task and there is ample room for model improvement.
APA
Liu, Z., Zhang, J., Lu, J., Ma, W. & Ho, J.C.. (2024). LogicPrpBank: A Corpus for Logical Implication and Equivalence. Proceedings of the 2024 AAAI Conference on Artificial Intelligence, in Proceedings of Machine Learning Research 257:57-65 Available from https://proceedings.mlr.press/v257/liu24a.html.

Related Material