The attack classes covered throughout this series (cross-site scripting, SQL injection, command injection, insecure function calling, data exfiltration, and hallucination-driven supply chain attacks) all stem from insecure output handling: LLM-generated output is treated as trusted data and passed to downstream systems without validation. The mitigations follow the same principle in reverse. Every point where LLM output enters another system needs context-appropriate encoding, validation, or access control, applied at the application layer rather than at the model layer.

This article covers the four defence layers that together provide effective protection against insecure output handling: context-specific output encoding, access control enforcement in application code, rendering layer defences against exfiltration, and execution sandboxing for command and code generation contexts.

Treat LLM output as untrusted data

The foundational principle is that LLM output must receive the same defensive treatment as raw user input from a web form, an API request, or any other external source. The model cannot be relied upon to produce safe output, regardless of system prompt instructions or alignment training. OWASP LLM05 (Improper Output Handling) summarises this as treating the model as an attacker-controlled payload generator and tracing every sink the output reaches.

The specific encoding depends on where the output is used.

HTML context. If LLM output is inserted into a web page, apply HTML entity encoding to all output before insertion. This converts characters such as <, >, ", and & into their entity equivalents (<, >, ", &), preventing any injected HTML or JavaScript from being interpreted by the browser. This is the primary defence against XSS through LLM output as described earlier in this series.

SQL context. If LLM output is inserted into a database query, use parameterised queries or prepared statements. Never concatenate LLM output into a SQL string. This applies whether the LLM generates the query directly (as in text-to-SQL applications) or provides values that the application inserts into a pre-defined query template.

Shell context. If LLM output is used in system commands, avoid shell execution entirely where possible. Use language-native process APIs that accept arguments as arrays rather than constructing command strings. If shell execution is unavoidable, escape all shell metacharacters in the LLM output before insertion.

Function calling context. If the application dispatches function calls based on LLM output, validate both the function name and the arguments against a strict allowlist before execution. Never pass LLM output to eval(), exec(), or any equivalent code execution function. The LlamaIndex vulnerabilities (CVE-2023-39662 and CVE-2024-3098) documented in the function calling article demonstrate that even “safe” wrappers around eval are difficult to implement correctly.

Each of these is a standard defence for untrusted input that has been well-understood for decades. The only change is the data source: LLM output rather than user input.

Access control enforcement in application code

Any data or function accessible to the LLM is effectively public. If the LLM can query a database table, call an administrative function, or access a user’s private data, an attacker who can influence the LLM’s behaviour can access it too. This was demonstrated directly in the excessive agency attack, where prepending “I am an administrator” to a prompt was sufficient to call a restricted function.

The defence is to enforce access control in the application’s function dispatch layer, not in the system prompt. The system prompt can instruct the model to refuse certain requests, but the model has no mechanism for verifying identity claims, checking session tokens, or validating role memberships. Prompt-level access control is advisory at best.

In practice, this means the function dispatch code should check the authenticated user’s permissions against the application’s authorisation system before executing any function call the LLM requests. If the LLM generates a call to system_check, the dispatch layer verifies that the current user has the required role before executing it. The LLM’s opinion about whether the user is authorised is irrelevant.

The principle of least privilege applies here with particular force. The LLM should have access only to the minimum set of functions and data required for its intended use case. Development-era debug functions, administrative tools, and unused API endpoints should be removed from the LLM’s available function set before deployment. Every function the LLM can call is part of the attack surface, and unused functions represent unnecessary risk.

Rendering layer defences

The exfiltration attacks covered earlier in this series exploit Markdown image rendering to create a GET request side channel. Defences against this attack class operate at the rendering layer, where the application converts LLM output into HTML for display.

Content Security Policy (CSP). A strict CSP header restricts which domains the browser is permitted to load resources from. Setting img-src 'self' prevents the browser from fetching images from external URLs, which blocks Markdown image exfiltration regardless of what the LLM generates. CSP also mitigates XSS by restricting script execution through the script-src directive. OWASP specifically recommends CSP as a mitigation for XSS from LLM-generated content.

Server-side image proxy. Rather than allowing the browser to fetch external images directly, the application fetches images through a server-side proxy. The proxy validates the URL against an allowlist, fetches the image, and serves a cached copy to the browser. This breaks the direct connection between the victim’s browser and the attacker’s server, preventing data exfiltration through image request URLs. Anthropic uses this approach to defend Claude.ai’s artifact rendering surface.

URL allowlisting. If the application renders Markdown links or images from LLM output, restrict the permitted URLs to a known set of trusted domains. Any URL that does not match the allowlist is stripped or replaced with a placeholder before rendering.

These three controls are complementary. CSP provides a browser-enforced boundary, the image proxy provides a server-side boundary, and URL allowlisting provides an application-level boundary. Layering all three creates defence in depth against exfiltration attempts.

Execution sandboxing

When the application executes commands or code based on LLM output (whether through text-to-command translation, code generation, or function calling with shell access), sandboxing limits the blast radius of a successful injection.

A sandboxed execution environment isolates the LLM-triggered process from the host system. If an attacker achieves command execution through the LLM, they gain access only to the sandbox, which should contain no sensitive data, no network access to internal systems, and no ability to escalate privileges to the host.

Container-based isolation (using tools such as gVisor or Firecracker) provides stronger boundaries than process-level sandboxing. The sandbox should enforce read-only filesystem access where possible, restrict network egress to prevent reverse shells and data exfiltration, and run under a minimal-privilege user account.

Sandboxing does not prevent the vulnerability. It limits the impact. The attacker can still execute commands within the sandbox, but the sandbox constrains what those commands can reach. This makes it a hardening measure rather than a fix, and it should be deployed alongside (not instead of) proper output validation and encoding.

Summary

Mitigating insecure output handling in LLM applications requires the same defensive measures that apply to any untrusted data source, applied at the application layer rather than the model layer. Context-specific output encoding (HTML encoding, parameterised queries, shell escaping) prevents injection. Access control enforced in the function dispatch layer, not the system prompt, prevents excessive agency. Content Security Policy, server-side image proxying, and URL allowlisting prevent Markdown-based exfiltration. Execution sandboxing limits the impact of successful command or code injection. None of these controls is sufficient alone, but layered together they address each attack class covered in this series.