DETECTING AND MITIGATING HALLUCINATIONS IN LARGE LANGUAGE MODELS: A REVIEW OF UNCERTAINTY, EVIDENCE GROUNDING, AND AGENTIC WORKFLOWS

Abstract: Large language models (LLMs) can give answers that sound smooth and believable but may still be wrong, unsupported, or self-contradictory. This becomes a bigger concern when LLMs are used in retrieval systems and agent- based workflows, where they may choose tools, understand user needs, search databases, and perform actions. In these situations, even one incorrect claim can affect later decisions.
This review looks at research on LLM truthfulness, hallucination detection, uncertainty, response consistency, retrieval- augmented generation (RAG), internal model signals, and agent reliability. It groups existing methods based on the evidence they use agreement between generated answers, prediction uncertainty, model-internal information, external retrieved sources, and workflow context.
One key point is that no single signal is enough to detect hallucinations. A model can be confident but wrong, retrieved information can be incomplete or misleading, and several answers may agree without being correct.
Therefore, reducing hallucinations requires combining different types of evidence.
To address this gap, the paper proposes a unified evidence-integration framework. The framework combines token-level probabilities and entropy, hidden-state and attention-based features where available, sensitivity to prompt changes, the quality and relevance of retrieved sources, claim-level support, and domain-specific context. In addition, it incorporates a confidence-score check and intent analysis using a classifier model to improve the reliability of the overall decision process.
Using this risk estimate, the system can answer directly, verify the response again, retrieve more information, ask the user for clarification, or avoid answering when confidence is low. The Model Context Protocol (MCP) is discussed to link agents with external tools and data, but it does not by itself ensure that the information is accurate. Because the project materials describe a proposed method rather than completed experiments, this review does not include new benchmark results. It ends by highlighting future evaluation areas such as detection accuracy, calibration, robustness, speed, cost, and safe action execution in agentic systems.

Keywords: large language models; hallucination detection; factuality; semantic entropy; uncertainty estimation; retrieval-augmented generation; Model Context Protocol; agentic AI; selective prediction