Prompt injection mitigations and defences

No single mitigation can eliminate prompt injection entirely. The non-deterministic nature of LLMs and the architectural inability to separate instructions from data mean that prompt injection mitigations reduce risk rather than remove it. This article covers the full spectrum of defensive approaches, from traditional techniques like prompt engineering and input filtering through to LLM-based defences including fine-tuning, adversarial prompt training, and guardrail models.

The fundamental limitation

The only guaranteed way to prevent prompt injection is to avoid using LLMs altogether. Every other mitigation is a trade-off between security, usability, cost, and complexity. This is not a counsel of despair. Layered defences can reduce the attack success rate dramatically. But anyone deploying an LLM-based application should understand that prompt injection is a residual risk that requires ongoing management, not a problem that can be solved once and forgotten.

Research published at ICLR 2025 demonstrated that adaptive attacks can achieve near-100% success rates even against models with advanced defences, reinforcing that no single layer of protection is sufficient in isolation.

Traditional mitigations

Prompt engineering

The most intuitive defence is to write a system prompt that instructs the model to reject malicious input. Adding rules like “never reveal the key” or “only respond to queries within this domain” can provide a baseline level of resistance against unsophisticated attacks.

However, as demonstrated throughout the earlier articles in this series, prompt engineering cannot prevent prompt injection. System prompt instructions are processed by the same attention mechanism as user input, and an attacker who can manipulate the user prompt can override or circumvent system prompt restrictions. Prompt engineering should be used to guide the model’s intended behaviour, not as a security control.

Filter-based approaches

Input filters scan user prompts for known malicious patterns before they reach the model. The simplest form is a blocklist that rejects prompts containing specific keywords or phrases associated with prompt injection (such as “ignore all previous instructions” or “DAN”). More sophisticated filters compare the user prompt against a database of known malicious prompts using similarity matching.

The limitations are predictable. Blocklists are trivially bypassed through synonyms, rephrasing, encoding, or language switching. Similarity-based filters perform better against known attacks but fail against novel techniques. Input length limits can reduce the attack surface for complex payloads but also constrain legitimate use.

Allowlists (restricting input to a predefined set of permitted queries) are theoretically the most secure filter approach, but they defeat the purpose of using an LLM. If the user can only ask a fixed set of questions, the answers might as well be hardcoded.

Filter-based mitigations are easy to implement and add minimal latency. They are useful as one layer in a defence-in-depth strategy but inadequate as a standalone protection.

Limiting LLM access and human oversight

The principle of least privilege applies to LLM deployments just as it does to traditional systems. If the model does not have access to sensitive data, an attacker cannot extract it through prompt injection. System prompts should never contain API keys, database credentials, or other secrets. Sensitive operations should be handled by backend systems that the LLM cannot reach directly.

Human oversight reduces the impact of successful prompt injection by keeping a human in the loop for consequential decisions. An LLM that recommends actions for human approval is significantly less dangerous than one that executes them autonomously. The indirect prompt injection example of an LLM automatically accepting or rejecting job applications illustrates why automated decision-making without human review creates unacceptable risk.

LLM-based mitigations

Fine-tuning for the target use case

Choosing an appropriate model and fine-tuning it for the specific deployment significantly narrows the attack surface. A model fine-tuned on customer support logs for a specific product is less likely to respond to off-topic prompt injection payloads than a general-purpose model, because its training distribution is concentrated on the target domain.

Fine-tuning does not eliminate prompt injection risk, but it reduces the model’s susceptibility by narrowing its scope of operation. A model that has learned to generate responses within a specific domain is harder to coerce into generating content outside that domain. As an added benefit, fine-tuned models typically produce higher-quality responses for their target use case.

Adversarial prompt training

Adversarial prompt training is one of the most effective mitigations available. The model is trained on datasets that include known prompt injection and jailbreak payloads, teaching it to recognise and refuse these patterns. The result is a model that has seen adversarial inputs during training and learned to handle them appropriately.

Major open-source models including Meta’s Llama and Google’s Gemma undergo adversarial prompt training as part of their standard training pipeline. This is why the latest versions of these models are significantly more resilient than their earlier releases. Organisations deploying open-source models benefit from this training without needing to conduct it themselves.

The limitation of adversarial prompt training is that it is reactive. The model can only refuse patterns similar to those it was trained on. Novel attack techniques that differ sufficiently from the training distribution can still succeed, which is why adversarial training needs to be repeated as new attack classes emerge.

Guardrail models

Guardrail models are separate, lightweight LLMs deployed alongside the main model to inspect input, output, or both. They add an independent layer of defence that operates outside the main model’s context window and is therefore not susceptible to the same prompt injection payload.

Input guards process the user prompt before it reaches the main model. They classify the input as benign or malicious. If the input is classified as malicious, it is blocked and the main model never sees it. Input guards can be fine-tuned specifically for prompt injection detection, giving them a narrower and more focused task than the main model.

Output guards process the main model’s response before it is returned to the user. They scan for evidence of successful prompt injection exploitation, including leaked system prompt content, harmful or restricted content, misinformation, or personally identifiable information. If the output guard detects a problem, the response is withheld and replaced with an error message.

Production guardrail systems include Meta’s LlamaGuardNVIDIA’s NeMo Guardrails, and LLM Guard, all of which provide configurable input and output filtering with support for custom detection rules.

The trade-off with guardrail models is increased complexity and computational cost. Depending on the configuration, one or two additional models must run for every request, increasing hardware requirements and response latency. Guardrail models are typically much smaller than the main model to minimise this overhead, but the cost is not zero.

Recent research on guardrail evasion has also shown that guardrail classifiers themselves can be bypassed using adversarial techniques such as character injection, homoglyphs, and zero-width characters. This means that guardrail models, like every other mitigation, are not a complete solution. They are most effective when combined with other defensive layers.

Summary

Prompt injection mitigations range from lightweight traditional approaches (prompt engineering, input filtering, access restriction) through to LLM-based defences (fine-tuning, adversarial prompt training, guardrail models). No single mitigation is sufficient. Prompt engineering cannot prevent attacks, filters are trivially bypassed, and even adversarial training and guardrail models are vulnerable to novel or adaptive techniques. The most effective defensive posture combines multiple layers, limits the LLM’s access to sensitive data and consequential actions, keeps humans in the loop for critical decisions, and treats prompt injection as an ongoing risk that requires continuous testing.

Leave a Reply

Your email address will not be published. Required fields are marked *

RELATED

Insecure output handling in LLM applications

LLM output is untrusted data. This article covers insecure output handling, the injection risks it enables, and where it sits…

LLM vulnerability scanning with garak

Garak is an open-source LLM vulnerability scanner that automates adversarial testing for prompt injection, jailbreaks, and encoding bypasses. Full walkthrough.

Introduction to LLM jailbreaking

LLM jailbreaking bypasses safety alignment to force models into generating restricted content. Covers DAN, roleplay, token smuggling, and adversarial suffixes.

Indirect prompt injection attack vectors

Indirect prompt injection embeds payloads in external data that LLMs process. Covers data poisoning, web content injection, email vectors, and…