Insecure Output Handlingin LLM Apps Explained
When you add a language model to an app, it is easy to think of its output as safe text. It is not. The model can be steered by hidden instructions in the data it reads, so its output is really just another kind of untrusted input. If you paste that output into a page, a query, or a command, you hand the attacker a way in. This guide explains the risk in plain words and shows how to stop it. It builds on our guide to the OWASP Top 10 for LLM applications.
What Is Insecure Output Handling?
Insecure output handling is when an application trusts the text a model produces and passes it on to another part of the system without checking or cleaning it. The model sits in the middle: it takes some input, and it returns some output. The danger is on the output side. If that output flows into a browser, a database, a command line, or another service as if it were safe, any harmful content inside it runs there. It is listed as LLM05 in the OWASP Top 10 for LLM applications.
Why Is the Output of a Model Dangerous?
A language model does not know the difference between a real instruction and a trick hidden in the data it reads. If a web page, a document, or a user message contains hidden instructions, the model may follow them and put attacker chosen content into its answer. This is called prompt injection. So the output you get back is not neutral text. It can contain a script, a command, or a query fragment that was placed there on purpose. Treating it as safe is the mistake.
How Is It Different From Prompt Injection?
The two go together but happen at different points. Prompt injection is about the input to the model. Insecure output handling is about what your app does with the output. Attackers often chain them: they inject through the input, then your app trusts the output and runs it.
| Aspect | Prompt injection | Insecure output handling |
|---|---|---|
| Where it happens | The input going into the model | The output coming out of the model |
| The core problem | The attacker changes what the model does | The app trusts what the model returns |
| Example | Hidden text tells the model to ignore its rules | Model output with a script tag runs in the browser |
| Main fix | Separate trusted instructions from untrusted data | Encode and validate output before you use it |
For the input side, see our guide on prompt injection. This article is about the output side, which is often forgotten.
What Attacks Can It Cause?
The damage depends on where the output goes. In every case, the output is treated as code or a command instead of plain text. These are the same bugs that hit normal web apps, which is why the fixes are familiar.
| Where the output goes | What can go wrong | How to stop it |
|---|---|---|
| A web page as HTML | Cross site scripting, so a script runs for other users | Encode the output for HTML and add a Content Security Policy |
| A database query | SQL injection | Use parameters, never build a query by joining text |
| A shell or command | Command injection, so the attacker runs commands on your server | Never pass model output to a shell; call safe APIs directly |
| A URL or outbound request | Server side request forgery, reaching internal systems | Validate and allowlist where requests can go |
| A file path | Path traversal, reading or writing the wrong file | Validate and normalise paths, and use safe file APIs |
The most common case is the first one. If you show a model's answer on a page without encoding it, you have created a cross site scripting hole. See cross site scripting and SQL injection for the classic versions of these bugs.
How Do You Prevent Insecure Output Handling?
- 1Treat every model output as untrusted input, exactly like text typed by a stranger.
- 2Encode the output for the exact place it goes: HTML encoding for pages, parameters for queries, and so on.
- 3Never send model output straight to a shell, eval, or a system command.
- 4Use parameters for every database query, never string building.
- 5Add a Content Security Policy as a backstop so a script that slips through cannot run.
- 6Expect a shape, not free text: if you asked for JSON, parse it safely and reject anything else.
- 7Give downstream tools the least power they need, so a bad output can do little.
- 8Log model outputs and the actions they trigger so you can review and catch abuse.
This connects to a wider idea: do not give your AI features more power than they need. See excessive agency in AI agents for how to limit what an AI can reach and do.
What Does OWASP Say About It?
OWASP lists this as LLM05, Improper Output Handling, in the Top 10 for LLM applications. The guidance is direct: treat the model as an untrusted source and apply the same output rules you would use for any user input. Encode for the target, use parameters for queries, avoid passing output to shells, and follow a zero trust approach between the model and the systems around it. In short, the model is not a trusted user, so its output is not trusted either.