PRISM: Calibrated Bayesian Fusion and Auditable Attribution for Reliable LLM Event Prediction from Text

Chengyuan Jin, Daojian Zeng, Kang Liu, Jun Zhao, Yubo Chen
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:2595-2617, 2026.

Abstract

We study event prediction from news text when intermediate drivers are not expert-annotated. Directly prompting large language models ({LLMs}) for forecast probabilities can yield miscalibrated outputs and explanations that are hard to audit. We propose the Probabilistic Reliability and Interpretability System, PRISM, which separates semantic extraction from probabilistic decision making. PRISM uses {LLMs} only to extract interpretable factors, treats repeated extractions as noisy measurements of latent states, and performs {Bayesian} inference to produce posterior predictive probabilities that propagate extraction and measurement uncertainty. For empirical reliability under temporal dependence, we add low-capacity post-hoc calibration and time-adaptive conformal prediction and report calibration and coverage diagnostics. PRISM also reports factor-contrast association summaries with uncertainty and explicit interpretation boundaries. On a {UCDP}–{GDELT} conflict benchmark, PRISM achieves AUROC 0.821 and {ECE} 0.058 on the test set; at 90% target coverage, it attains empirical coverage 0.908 with average set size 1.38.

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-jin26a, title = {PRISM: Calibrated {Bayesian} Fusion and Auditable Attribution for Reliable {LLM} Event Prediction from Text}, author = {Jin, Chengyuan and Zeng, Daojian and Liu, Kang and Zhao, Jun and Chen, Yubo}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {2595--2617}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/jin26a/jin26a.pdf}, url = {https://proceedings.mlr.press/v337/jin26a.html}, abstract = {We study event prediction from news text when intermediate drivers are not expert-annotated. Directly prompting large language models ({LLMs}) for forecast probabilities can yield miscalibrated outputs and explanations that are hard to audit. We propose the Probabilistic Reliability and Interpretability System, PRISM, which separates semantic extraction from probabilistic decision making. PRISM uses {LLMs} only to extract interpretable factors, treats repeated extractions as noisy measurements of latent states, and performs {Bayesian} inference to produce posterior predictive probabilities that propagate extraction and measurement uncertainty. For empirical reliability under temporal dependence, we add low-capacity post-hoc calibration and time-adaptive conformal prediction and report calibration and coverage diagnostics. PRISM also reports factor-contrast association summaries with uncertainty and explicit interpretation boundaries. On a {UCDP}–{GDELT} conflict benchmark, PRISM achieves AUROC 0.821 and {ECE} 0.058 on the test set; at 90% target coverage, it attains empirical coverage 0.908 with average set size 1.38.} }
Endnote
%0 Conference Paper %T PRISM: Calibrated Bayesian Fusion and Auditable Attribution for Reliable LLM Event Prediction from Text %A Chengyuan Jin %A Daojian Zeng %A Kang Liu %A Jun Zhao %A Yubo Chen %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-jin26a %I PMLR %P 2595--2617 %U https://proceedings.mlr.press/v337/jin26a.html %V 337 %X We study event prediction from news text when intermediate drivers are not expert-annotated. Directly prompting large language models ({LLMs}) for forecast probabilities can yield miscalibrated outputs and explanations that are hard to audit. We propose the Probabilistic Reliability and Interpretability System, PRISM, which separates semantic extraction from probabilistic decision making. PRISM uses {LLMs} only to extract interpretable factors, treats repeated extractions as noisy measurements of latent states, and performs {Bayesian} inference to produce posterior predictive probabilities that propagate extraction and measurement uncertainty. For empirical reliability under temporal dependence, we add low-capacity post-hoc calibration and time-adaptive conformal prediction and report calibration and coverage diagnostics. PRISM also reports factor-contrast association summaries with uncertainty and explicit interpretation boundaries. On a {UCDP}–{GDELT} conflict benchmark, PRISM achieves AUROC 0.821 and {ECE} 0.058 on the test set; at 90% target coverage, it attains empirical coverage 0.908 with average set size 1.38.
APA
Jin, C., Zeng, D., Liu, K., Zhao, J. & Chen, Y.. (2026). PRISM: Calibrated Bayesian Fusion and Auditable Attribution for Reliable LLM Event Prediction from Text. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:2595-2617 Available from https://proceedings.mlr.press/v337/jin26a.html.

Related Material