Magic Numbers: An Algorithm for Mitigating Hallucinations in LLMs'

doi:10.1007/978-3-032-03711-4_2

Magic Numbers: An Algorithm for Mitigating Hallucinations in LLMs'

2026-01-01

0

PRE

AI

H

Hallstrom, Hasse J. *

DOI：10.1007/978-3-032-03711-4_2

Original

Original request for help

Save

Abstract

En 中文

Large language models (LLMs) sometimes generate convincingly written but factually incorrect content, commonly referred to as hallucinations. We introduce an algorithm, Magic Numbers (MN), which is compatible with any LLM or LLM API that outputs token probabilities (at the time of writing, only certain GPT-models). It uses combinations of perplexity and logit entropy to detect and fix hallucinations in real time. We show that MN can detect hallucinations in GPT-4o with only a negligible addition to the computational cost. Our experiments use two frameworks for hallucination measurement. HaluBench (which focusses on question-answering) and HHEM (which targets summarization tasks). The results indicate that Magic Numbers can improve the veracity of GPT-4o in various question-answer tasks and also improve the factual consistency in text summarisation.

Keywords:

Hallucinations

Large Language Models

Perplexity Gating

Entropy Gating

GPT

Magic Numbers: An Algorithm for Mitigating Hallucinations in LLMs'

Abstract

Journal

Organization