LLM hallucinations are instances where a model generates fabricated, misleading, or factually incorrect output while presenting it with apparent confidence. Unlike the injection vulnerabilities covered earlier in this series, hallucinations are not caused by an attacker exploiting a code path. They are an inherent property of probabilistic token generation. The security impact arises when applications treat hallucinated output as trusted data and act on it without validation, whether that means displaying fabricated policy information to customers, inserting buggy code into a codebase, or installing a dependency that does not exist.

This article covers the types of hallucination, why they occur, the financial and technical security risks they create (including the supply chain attack class known as slopsquatting), and the mitigation strategies available at the model, application, and deployment layers.

Types of hallucination

Hallucinations fall into three categories based on what the generated output contradicts.

Fact-conflicting hallucination produces output that contradicts verifiable facts. A model stating that a word contains three instances of a letter when it contains two is a fact-conflicting hallucination. Fabricated citations, invented statistics, and incorrect technical specifications fall into this category.

Input-conflicting hallucination produces output that contradicts information provided in the prompt. If the prompt states “My shirt is red” and the model responds “Your shirt is blue”, the model is hallucinating against its own input context.

Context-conflicting hallucination produces output that contradicts itself within the same response or conversation. The model may correctly state a fact in one sentence and contradict it in the next, or confuse entities it introduced earlier in the response. This type is common in lengthy or multi-turn interactions where the model loses coherence over the span of the generated text.

All three types are difficult to detect because hallucinated output is typically fluent, well-structured, and presented with the same apparent confidence as factually correct output. The model provides no signal that distinguishes a hallucinated response from a grounded one.

Why hallucinations occur

Hallucination is inherent to how LLMs generate text. The model predicts the next token based on statistical patterns in its training data, not based on a grounded understanding of truth. Several factors influence the likelihood of hallucination.

Training data quality is the most fundamental factor. Incomplete datasets leave gaps in the model’s knowledge, while noisy or biased data teaches the model incorrect patterns that it reproduces at inference time. Prompt design also contributes: ambiguous, contradictory, or underspecified prompts increase the probability that the model fills in gaps with fabricated information rather than expressing uncertainty.

Because hallucination is a structural property of the generation process rather than a bug that can be patched, it cannot be eliminated entirely. It can only be reduced through better training data, fine-tuning, and application-level mitigations, and its impact managed through validation of the model’s output before it reaches downstream systems or users.

Financial and legal liability

Hallucinations can cause direct financial harm when applications present hallucinated output to users without validation.

The most widely cited case is Moffatt v. Air Canada (2024 BCCRT 149). A passenger used Air Canada’s website chatbot to research bereavement fare policies after a family death. The chatbot stated that bereavement rates could be applied retroactively within 90 days of ticket purchase. This was incorrect: Air Canada’s actual policy required the discounted fare to be applied before purchase. After the passenger’s refund request was denied, they filed a claim with British Columbia’s Civil Resolution Tribunal.

Air Canada argued it could not be held liable for information provided by the chatbot. The tribunal rejected this defence, ruling that a company is responsible for all information on its website regardless of whether it comes from a static page or a chatbot. Air Canada was ordered to pay CAN$812.02 in damages and fees.

The financial amount was small, but the precedent was significant. The ruling established that deploying an LLM chatbot on a commercial website does not create a liability shield. The organisation is responsible for the accuracy of the chatbot’s output to the same standard as any other published information. Any company deploying customer-facing LLM applications without output validation is exposed to the same risk.

Slopsquatting and supply chain attacks

Hallucinated package names in LLM-generated code create a supply chain attack vector known as slopsquatting. When a model generates a code snippet that imports a package that does not exist, an attacker can register that package name on a public repository (PyPI, npm, or similar) and publish malicious code under it. Any developer who installs the hallucinated dependency executes the attacker’s payload.

Consider a model that generates the following Python snippet in response to a coding question.

from hacktheboxsolver import solve

solve('Blazorized')

The package hacktheboxsolver does not exist. If a developer runs pip install hacktheboxsolver, the installation fails. But if an attacker has already registered the name on PyPI and published a package containing malware, the installation succeeds and the malicious code executes in the context of the developer’s user account.

What makes slopsquatting dangerous at scale is that hallucinated package names are repeatable. A 2024 study across 16 code-generation models found that approximately 5.2% of packages recommended by commercial models and 21.7% by open-source models were hallucinated, producing 205,474 unique fictitious package names across 756,000 code samples. A 2026 re-evaluation of five frontier models found that while hallucination rates had narrowed to roughly 4.6% to 6.1%, 127 package names were hallucinated identically across all five models. This cross-model consistency means attackers can probe one model to discover hallucinated names and then register them, knowing that developers using any of the other models will encounter the same recommendations.

In July 2025, PyPI disclosed that its administrators had been alerted by a user whose LLM recommended installing a project that did not exist on the registry. The attack vector is no longer theoretical.

Mitigation strategies

Because hallucination cannot be eliminated, mitigations operate at multiple layers.

At the training and model layer, using high-quality training data from credible sources reduces the frequency of fact-conflicting hallucinations. Fine-tuning a general-purpose model on domain-specific data further reduces hallucination within that domain by exposing the model to accurate, domain-relevant patterns.

At the application layer, retrieval-augmented generation (RAG) enriches the user’s query with relevant information from an external knowledge base before generation. This grounds the model’s response in retrieved facts rather than relying solely on parametric memory, reducing (but not eliminating) the likelihood of fabrication.

At the output validation layer, three approaches attempt to measure the model’s certainty before accepting a response. Logit-based measurement evaluates token-level probabilities from the model’s internal state, but requires access to model internals and is unavailable for closed-source models. Verbalize-based measurement prompts the model to self-report a confidence score, but LLMs are unreliable estimators of their own certainty. Consistency-based measurement generates multiple responses to the same prompt and compares them, operating on the principle that factual responses are more likely to be consistent across generations than hallucinated ones.

A multi-agent approach uses multiple LLMs to debate their responses and converge on a consensus, reducing the likelihood that a single model’s hallucination goes unchallenged.

None of these approaches is sufficient on its own. The most effective mitigation is human validation: a review process that checks LLM output for correctness before it is published, executed, or acted upon. For code generation specifically, this means treating LLM-generated code with the same scrutiny as code from an untrusted external contributor, including dependency verification against the actual contents of public package registries.

Summary

LLM hallucinations are an inherent property of probabilistic text generation, not a defect that can be patched. Their security impact ranges from financial liability (as established in the Air Canada tribunal ruling) to supply chain compromise through slopsquatting, where attackers register hallucinated package names on public repositories and wait for developers to install them. Mitigation requires a layered approach: high-quality training data, RAG-based grounding, output certainty measurement, and human validation before LLM output is trusted by downstream systems or users.