What Is a Hallucination, and Why Should Your Board Care?

Large Language Models generate responses by predicting the most likely continuation of a conversation based on patterns in training data. They do not retrieve verified facts from a database. When asked about something specific (a case citation, a regulation number, a financial figure, a person's biography), the model generates text that looks like an answer, whether or not it corresponds to reality.

The model has no mechanism for distinguishing between things it knows reliably and things it is confabulating. It produces both with equal confidence and fluency. This is different from a human expert who might say 'I am not certain about that, let me check.' AI systems, unless specifically prompted to express uncertainty, tend to present all their outputs with similar apparent confidence.

Hallucinations range from subtle to egregious. At the subtle end, an AI might slightly misquote a statistic, attribute a quote to the wrong person, or describe a regulation that exists but get a specific detail wrong. These errors are particularly dangerous because they are hard to detect without domain knowledge.

At the egregious end, AI systems have fabricated entire legal cases (complete with judges' names, dates, and citations), invented academic papers that do not exist, and generated plausible-sounding but entirely fictional biographical details about real people. The 2023 case where a US attorney submitted AI-generated legal citations to a court, only to discover the cases did not exist, is now a landmark case study in the consequences of hallucination in professional contexts.

For executives, the business risk from AI hallucinations concentrates in contexts where specific factual accuracy is required and the output will be acted on without independent verification.

Legal contexts: AI-generated contract summaries, regulatory interpretations, or compliance assessments that contain hallucinated details can lead to decisions based on non-existent legal authority.

Financial contexts: AI-generated market data, financial calculations, or financial projections that incorporate hallucinated figures create financial risk and potential liability.

Customer-facing contexts: AI systems that tell customers incorrect information about products, policies, pricing, or warranty terms create consumer protection liability.

Reputational contexts: AI-generated content that contains incorrect factual claims, particularly about people or organisations, creates defamation risk.

The board-level question is not whether hallucinations occur (they do, in all current AI systems) but whether the governance framework around AI deployments requires adequate verification of AI outputs in contexts where hallucination risk is material.

The most effective technical approach to reducing hallucination risk is Retrieval-Augmented Generation (RAG), which connects the AI to verified source documents and requires it to base responses on those documents rather than on training data. When the AI is working from a specific document it has been given, rather than from pattern-matched training, hallucination rates drop significantly.

The most important governance approach is requiring human expert review of AI outputs before they are used in high-stakes contexts. This is not a workaround for a flaw that will eventually be fixed. It is the appropriate governance response to a persistent characteristic of current AI technology.