When Do LLMs Listen? Confidence-Guided Knowledge Acceptance in LLMs

Previous work shows that injecting external knowledge from Knowledge Graphs (KGs) can improve reasoning in Large Language Models on multiple-choice question answering. KGs provide structured factual knowledge that reduces errors and hallucinations without costly model updates. Most studies focus on which knowledge to extract from KGs and how to represent it in prompts to improve task accuracy. In contrast, this study examines knowledge acceptance in LLMs, investigating when models use, ignore, or resist injected knowledge. We introduce a confidence-guided framework that categorizes predictions as high, moderate, or low certainty. High certainty indicates a strong preference for a single answer, moderate reflects several plausible options with similar probabilities, and low corresponds to diffuse predictions with no clear preference. To study knowledge injection, we introduce KG-derived statements into the model’s context and track changes in prediction confidence. Interventions include supportive knowledge (reinforcing the model’s top choice), opposing knowledge (favoring alternatives), and irrelevant or noisy statements. Our analysis reveals consistent patterns: highly confident predictions largely ignore new evidence, while moderate and low-confidence predictions are more sensitive, with the model switching between similarly probable options. Low-confidence choices may gain probability but rarely overturn the initial decision. The model remains robust to noisy or irrelevant information as long as relevant knowledge dominates the context.