Treat Model Output Like User Input
Rendering generated text as HTML is XSS where the payload is written by a stranger through your own feature.
We spent twenty years learning not to trust user input. Then we added a feature that generates text influenced by user input and rendered it straight into the page.
Model output is untrusted input. It arrives from a system a stranger can influence, and it lands in exactly the sinks that have always been dangerous.
The vulnerability in one line
// an XSS vector where the payload is authored via your own AI feature
messageEl.innerHTML = await getAssistantReply(userMessage)Ask the assistant to include a specific string, and it obliges. If that string is markup and you render it as HTML, script runs in your users' sessions.
The same applies wherever generated text meets an interpreter: a SQL string, a shell command, a template, a redirect URL, a file path.
The fixes
Render as text. In Vue, {{ reply }} is safe and v-html is not. React's default is safe; dangerouslySetInnerHTML is the one to justify.
If you need formatting, convert markdown to HTML through a renderer you control, then sanitise the result with an allowlist. Bold, lists, links, code — nothing else. Check link protocols explicitly, because javascript: in a generated link is a real finding.
Never interpolate into a query. Parameterised statements only. If output must reach a database, treat it exactly as you would a request body.
Validate structured output against a schema. If you asked for JSON, parse it and validate the shape. Do not trust that the fields you expected are the fields you got, and never eval a response.
Constrain what output can trigger. If a reply drives an action, the action is authorised server-side against the session — never against something the model asserted.
Beyond the technical
There is a category of output risk that is not a vulnerability but is still a problem: a client's assistant confidently stating something false about pricing, availability, or a legal position.
For that, the controls are a tight scope in the system prompt, refusal instructions for out-of-scope questions, and never letting generated text state a commitment the business has not made. On a booking assistant, the model may describe services; only the server confirms a booking.
Tooling
Libraries like LLM Guard bundle output scanners — toxicity, leakage, format checks — and are useful as a layer. They do not replace sanitising at the render site, which is where the actual vulnerability lives.
Resources
- Repo: protectai/llm-guard
- Reference: OWASP LLM Top 10
- Video walkthroughs: YouTube: insecure output handling LLM
Need this built properly?
I build secure, fast, bilingual platforms for clients across Egypt, Saudi Arabia, the UAE and Kuwait.
