Why It Matters
For agents, this is the confused-deputy problem in natural language. The model acts with the user's authority while consuming text controlled by other parties.
Problem
- Early prompt-injection discussions mostly assumed the attacker directly prompts their own model session.
- LLM-integrated applications increasingly retrieve webpages, documents, emails, code, and search results at inference time.
- Agents and application-integrated LLMs can call tools or APIs on behalf of a user.
- Existing input/output filters do not cleanly distinguish untrusted retrieved data from trusted instructions.
Key gap: The paper asks what happens when the attacker is not the user, but controls data that the user's LLM application will later retrieve and process.
Method
Model LLM applications as systems that retrieve untrusted text and call tools, classify indirect injection delivery and impacts, then demonstrate attacks against controlled synthetic applications and real-world systems.
- Define indirect prompt injection as malicious prompt delivery through retrieved external data.
- Map injection methods: passive retrieval, active delivery such as email, user-driven copy/paste, hidden content, encoded payloads, and multi-stage retrieval.
- Map threat classes including data theft, remote control, persistence, worming, fraud, disinformation, and availability attacks.
- Build synthetic LLM applications with tools such as search, view, URL retrieval, email, address book, and memory.
- Test attacks on synthetic applications and black-box real-world systems such as Bing Chat and code-completion contexts.
- Discuss limitations, disclosure, and mitigations while noting that robust defenses remain unresolved.
Claims
C1 · paper-supported
LLM-integrated applications blur the line between data and instructions when they retrieve untrusted content into the model context.
Scope: Abstract and Introduction.
- E-data-instruction: The authors argue that retrieval-augmented LLM applications blur data and instruction boundaries, making untrusted external text capable of steering model behavior.
C2 · paper-supported
Indirect prompt injection enables remote attackers without direct model access to influence a victim's LLM application by poisoning likely retrieved data.
Scope: Abstract and Section 3.
- E-indirect-attack: Adversaries can place prompts in data likely to be retrieved, remotely affecting a victim's application without directly prompting it.
C3 · paper-supported
The attack surface includes passive retrieval, active delivery through channels like email, user-driven delivery, hidden prompts, encoded prompts, and multi-stage payloads.
Scope: Section 3.1 and Section 4.3.
- E-injection-methods: The paper describes passive retrieval, active email-like delivery, user-driven injection, hidden prompt content, multi-stage payloads, and Base64/encoded injections.
C4 · paper-supported
Indirect prompt injection can enable remote control, data theft, persistence, worming, social engineering, misinformation, and availability attacks in LLM-integrated systems.
Scope: Abstract, Section 3.2, and repository demonstrations.
- E-impact-taxonomy: The paper and repository describe impacts including remote control, leaking user data, persistent compromise, spreading injections, social engineering, code-completion attacks, and availability disruption.
C5 · paper-supported
The paper demonstrates attacks in synthetic GPT-3/GPT-4-style applications and against real-world systems including Bing Chat and code-completion contexts.
Scope: Abstract and Section 4.
- E-demonstrations: The authors build synthetic applications with search, view, URL retrieval, email, address book, and memory tools, and test attacks against Bing Chat and code-completion contexts.
C6 · paper-supported
When the LLM controls tool/API calls, retrieved prompts can manipulate not only generated text but also the timing, content, and direction of external API use.
Scope: Abstract and Sections 3-4.
- E-tool-control: The authors show that processing retrieved prompts can manipulate application functionality and control whether and how APIs are called.
C7 · paper-supported
The paper argues that effective mitigations were lacking and that RLHF, filtering, LLM supervisors, and retrieved-input scanning are brittle or unresolved defenses.
Scope: Section 5.6.
- E-mitigations: The paper discusses RLHF, input/output filtering, retrieved-input filtering, LLM supervisors, and interpretability-style outlier detection, concluding that robust defenses and their resilience to obfuscation require more work.
Agent Infrastructure Takeaways
- Treat retrieved content as hostile data, never as authority-bearing instructions.
- A model should not be the only boundary between untrusted text and privileged tools.
- Tool calls need capability scoping, confirmation, policy checks, audit logs, and least privilege independent of prompt intent.
- Memory can make prompt injection persistent; stored notes and retrieved history need the same distrust as webpages or emails.
- Defenses should be architectural: data provenance, content labeling, separate planning/execution authority, sandboxed tools, and explicit user approval for side effects.
- Prompt injection is closer to confused-deputy and capability-security failure than to ordinary bad prompting.
Caveats
- Many demonstrations are qualitative proof-of-concepts rather than statistically measured benchmark results.
- The real-world systems tested were black boxes and have changed since 2023.
- The paper intentionally avoided seeding public sources with live malicious prompts for ethical reasons.
- Some mitigations and product defenses have improved since publication, but no general clean fix is established by the paper.
- The attack requires delivery: hostile content must be retrieved, included in context, and granted enough influence to affect output or tools.
- Impact depends on connected privileges; read-only summarizers differ from agents that can send email, write memory, or call APIs.