<!-- .slide: class="title-slide apple-title" --> <div class="slide-frame slide-frame--center"> <p class="deck-kicker">agent-readings / Security, Safety, and Trust</p> <h1>GCG Attacks</h1> <p class="deck-subtitle">Alignment is not adversarial robustness</p> <div class="deck-meta-strip"> <div class="deck-meta-pill"><strong>2023</strong><span>arXiv 2023</span></div> <div class="deck-meta-pill"><strong>7</strong><span>claims</span></div> <div class="deck-meta-pill"><strong>7</strong><span>evidence refs</span></div> <div class="deck-meta-pill"><strong>paper-read</strong><span>status</span></div> </div> </div> --- <!-- .slide: class="statement-slide" --> <div class="slide-frame"> <p class="deck-kicker">The one sentence</p> <h2>GCG showed that adversarial suffixes can be automatically optimized on open models and transfer to other aligned LLMs, including black-box systems.</h2> <p class="deck-note">For agents, this means model refusal is not a hard security boundary. Tool authority, sandboxing, policy checks, and output controls must not rely solely on the model staying aligned under adversarial input.</p> </div> --- <!-- .slide: class="problem-slide" --> <div class="slide-frame"> <p class="deck-kicker">Before this paper</p> <h2>The friction was structural.</h2> <div class="apple-card-grid apple-card-grid--three"> <div class="apple-card"><p>Manual jailbreaks existed but required human ingenuity and were often brittle or patchable.</p></div> <div class="apple-card"><p>Automatic adversarial prompt generation had limited success against aligned chat models.</p></div> <div class="apple-card"><p>LLM inputs are discrete tokens, making direct gradient-based optimization harder than image perturbation.</p></div> </div> <div class="deck-callout"> <span>Key gap</span> <p>The paper asks whether adversarial prompts for aligned LLMs can be generated automatically, made universal across many requests, and transferred to other models.</p> </div> </div> --- <!-- .slide: class="idea-slide" --> <div class="slide-frame"> <p class="deck-kicker">Core idea</p> <h2>Search over suffix tokens on surrogate open models to maximize a refusal-bypassing objective, then evaluate whether the resulting suffixes work across prompts,...</h2> <p class="deck-note">Search over suffix tokens on surrogate open models to maximize a refusal-bypassing objective, then evaluate whether the resulting suffixes work across prompts, models, and black-box public interfaces.</p> <div class="idea-loop" aria-label="Agent loop"> <div><strong>Model</strong><span>reason</span></div> <div><strong>Runtime</strong><span>act</span></div> <div><strong>World</strong><span>observe</span></div> </div> <div class="step-strip"> <div class="step-item"> <span>01</span> <p>Define harmful-string and harmful-behavior evaluation settings in AdvBench.</p> </div> <div class="step-item"> <span>02</span> <p>Optimize suffixes against open surrogate models using a combined greedy and gradient-guided discrete search.</p> </div> <div class="step-item"> <span>03</span> <p>Scale from single behavior/model attacks to universal suffixes over multiple behaviors.</p> </div> <div class="step-item"> <span>04</span> <p>Optimize across multiple surrogate models to improve transfer.</p> </div> </div> </div> --- <!-- .slide: class="grammar-slide" --> <div class="slide-frame"> <p class="deck-kicker">Action grammar</p> <h2>The agent is an interface contract.</h2> <div class="apple-card-grid apple-card-grid--two"> <div class="apple-card"><p>base request: the original user request under test.</p></div> <div class="apple-card"><p>suffix: modifiable token sequence appended to the request.</p></div> <div class="apple-card"><p>objective: probability of a targeted affirmative response prefix rather than refusal.</p></div> <div class="apple-card"><p>candidate substitution: a possible token replacement proposed from gradient information.</p></div> </div> </div> --- <!-- .slide: class="code-slide" --> <div class="slide-frame"> <p class="deck-kicker">Executable shape</p> <h2>The mechanism should fit on one screen.</h2> ```python defensive_summary_only = true # The review intentionally omits attack strings and runnable optimization details. # Conceptually: optimize a suffix on surrogate models, then measure transfer ASR on held-out requests and models. ``` </div> --- <!-- .slide: class="proof-slide" --> <div class="slide-frame"> <p class="deck-kicker">Evidence</p> <h2>The proof objects.</h2> <div class="proof-grid"> <div class="proof-card"> <span>E-gcg-method</span> <p>The paper defines Greedy Coordinate Gradient as using token-level gradients to identify promising replacements and exact forward evaluation to choose greedy suffix updates.</p> </div> <div class="proof-card"> <span>E-universal-method</span> <p>The paper optimizes one suffix over multiple prompts and aggregates losses/gradients across multiple models to improve universality and transfer.</p> </div> <div class="proof-card"> <span>E-advbench</span> <p>AdvBench contains harmful-string and harmful-behavior settings and uses attack success rate to evaluate exact target strings or attempted compliance with harmful instructions.</p> </div> <div class="proof-card"> <span>E-single-model-results</span> <p>GCG achieves strong results in single-model attacks, including 88% harmful-string success on Vicuna-7B and higher held-out behavior success than AutoPrompt on LLaMA-2-7B-Chat in...</p> </div> </div> </div> --- <!-- .slide: class="claim-slide" --> <div class="slide-frame"> <p class="deck-kicker">Claim map</p> <h2>What the review actually supports.</h2> <div class="claim-grid"> <div class="claim-card"> <span>C1 · paper-supported</span> <p>GCG combines gradient-guided token candidate selection with greedy evaluation to optimize discrete adversarial suffixes.</p> <small>E-gcg-method</small> </div> <div class="claim-card"> <span>C2 · paper-supported</span> <p>The paper extends suffix optimization to universal multi-prompt and multi-model attacks.</p> <small>E-universal-method</small> </div> <div class="claim-card"> <span>C3 · paper-supported</span> <p>AdvBench evaluates both harmful strings and harmful behaviors, with attack success rate as the primary metric.</p> <small>E-advbench</small> </div> <div class="claim-card"> <span>C4 · paper-supported</span> <p>On single-model experiments, GCG substantially outperforms prior automatic prompt-generation baselines such as PEZ, GBDA, and AutoPrompt.</p> <small>E-single-model-results</small> </div> </div> </div> --- <!-- .slide: class="takeaway-slide" --> <div class="slide-frame"> <p class="deck-kicker">Agent infrastructure</p> <h2>What changes if you build systems.</h2> <div class="apple-card-grid apple-card-grid--two"> <div class="apple-card"><p>Do not use model refusal as the only security boundary for a tool-using agent.</p></div> <div class="apple-card"><p>Adversarial prompt testing should be part of agent evaluation, especially before granting tools, memory writes, or external side effects.</p></div> <div class="apple-card"><p>Transfer attacks mean closed-source or black-box deployment does not eliminate risk if similar open models can be used as surrogates.</p></div> <div class="apple-card"><p>Safety filters should be layered with capability scoping, sandboxing, output validation, policy enforcement, and human approval for sensitive actions.</p></div> </div> </div> --- <!-- .slide: class="caveat-slide" --> <div class="slide-frame"> <p class="deck-kicker">Caveats</p> <h2>What this does not prove.</h2> <div class="apple-card-grid apple-card-grid--two"> <div class="apple-card"><p>The paper intentionally studies objectionable-content elicitation, not every possible alignment objective.</p></div> <div class="apple-card"><p>Attack success depends on model version, interface, decoding parameters, and provider patches; reported numbers are historical.</p></div> <div class="apple-card"><p>The review intentionally omits attack strings and runnable optimization details.</p></div> <div class="apple-card"><p>Harmful-behavior evaluation involves human judgment about attempted compliance.</p></div> </div> </div> --- <!-- .slide: class="closing-slide" --> <div class="slide-frame"> <p class="deck-kicker">Run it</p> <h2>No PoC yet.</h2> <p class="deck-note">No PoC yet. This repo should avoid publishing jailbreak-generation PoCs; a safe contribution could instead implement a benign robustness harness that checks refusal consistency on harmless canary prompts.</p> <div class="reference-list"> <ul><li>Paper: <a href="https://arxiv.org/abs/2307.15043">Universal and Transferable Adversarial Attacks on Aligned Language Models</a></li><li>Project/code: <a href="https://llm-attacks.org/">https://llm-attacks.org/</a></li><li>complementary_attack_surface: <a href="https://arxiv.org/abs/2302.12173">Indirect Prompt Injection</a></li><li>risk_testing: <a href="https://arxiv.org/abs/2309.15817">ToolEmu</a></li><li>trust_boundary: <a href="https://dl.acm.org/doi/10.1145/358198.358210">Reflections on Trusting Trust</a></li></ul> </div> </div> Note: This deck is synthesized from `data/reviews/gcg-zou-2023.json`. Update the review record, then run `bun run build`.