agent-readings

<div class="slide-frame slide-frame--center">
  <p class="deck-kicker">agent-readings / Start Here</p>
  <h1>CoALA</h1>
  <p class="deck-subtitle">Memory, action, and decision loops as the agent architecture map</p>

<div class="deck-meta-strip">
    <div class="deck-meta-pill"><strong>2023</strong><span>Transactions on Machine Learning Research, 2024</span></div>
    <div class="deck-meta-pill"><strong>6</strong><span>claims</span></div>
    <div class="deck-meta-pill"><strong>6</strong><span>evidence refs</span></div>
    <div class="deck-meta-pill"><strong>paper-read</strong><span>status</span></div>
  </div>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">The one sentence</p>
  <h2>CoALA turns language agents into a cognitive-architecture design problem by separating memory modules, internal and external actions, and a looping decision procedure.</h2>
  <p class="deck-note">For agent infrastructure, CoALA names the runtime pieces that need explicit interfaces: working state, long-term stores, tool/environment grounding, retrieval, reasoning, learning, and the top-level decision loop.</p>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Before this paper</p>
  <h2>The friction was structural.</h2>
  <div class="apple-card-grid apple-card-grid--three">
    <div class="apple-card"><p>Early language agents often placed an LLM directly in a feedback loop with an external environment.</p></div>
<div class="apple-card"><p>Newer agents added reasoning, planning, reflection, retrieval, and memory, but papers used inconsistent names for similar mechanisms.</p></div>
<div class="apple-card"><p>Traditional cognitive architectures had mature notions of working memory, long-term memory, grounding, learning, and decision cycles, but required hand-specified symbolic rules.</p></div>
  </div>
  <div class="deck-callout">
    <span>Key gap</span>
    <p>The paper addresses the lack of a systematic framework for comparing existing language agents and planning new ones at the architectural level.</p>
  </div>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Core idea</p>
  <h2>Map language agents onto a cognitive architecture: keep active state in working memory, persist experience and knowledge in long-term memories, expose...</h2>
  <p class="deck-note">Map language agents onto a cognitive architecture: keep active state in working memory, persist experience and knowledge in long-term memories, expose grounding/retrieval/reasoning/learning as actions, and run a repeated decision cycle that plans before...</p>
  <div class="idea-loop" aria-label="Agent loop">
    <div><strong>Model</strong><span>reason</span></div>
    <div><strong>Runtime</strong><span>act</span></div>
    <div><strong>World</strong><span>observe</span></div>
  </div>
  <div class="step-strip">
    <div class="step-item">
  <span>01</span>
  <p>Frame LLM calls and prompt chains as production-like operations over text.</p>
</div>
<div class="step-item">
  <span>02</span>
  <p>Import cognitive-architecture concepts to define memory modules and action types.</p>
</div>
<div class="step-item">
  <span>03</span>
  <p>Classify recent agents by their long-term memory, external grounding, internal actions, and decision-making procedure.</p>
</div>
<div class="step-item">
  <span>04</span>
  <p>Use the classification to identify reusable abstractions and underexplored design directions.</p>
</div>
  </div>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Action grammar</p>
  <h2>The agent is an interface contract.</h2>
  <div class="apple-card-grid apple-card-grid--two">
    <div class="apple-card"><p>Grounding action: external interaction such as robot control, web navigation, API use, or communication.</p></div>
<div class="apple-card"><p>Retrieval action: read from long-term memory into working memory.</p></div>
<div class="apple-card"><p>Reasoning action: use the LLM to update working memory with inferred facts, plans, critiques, or candidate actions.</p></div>
<div class="apple-card"><p>Learning action: write experience, knowledge, code, prompts, or parameters into long-term memory.</p></div>
  </div>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Executable shape</p>
  <h2>The mechanism should fit on one screen.</h2>

```python
while true:
    working_memory.update(observe_environment())
    candidates = reason_and_retrieve(working_memory, long_term_memory)
    selected = select_action(candidates)
    if selected.kind == "grounding":
        working_memory.update(environment.step(selected))
    elif selected.kind == "learning":
        long_term_memory.write(selected)
```
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Evidence</p>
  <h2>The proof objects.</h2>
  <div class="proof-grid">
    <div class="proof-card">
  <span>E-framework-definition</span>
  <p>The paper states that CoALA organizes agents by information storage, action space, and decision-making procedure, then develops those concepts in Section 4.</p>
</div>
<div class="proof-card">
  <span>E-memory-taxonomy</span>
  <p>CoALA defines short-term working memory and long-term episodic, semantic, and procedural memories. Working memory persists active state across LLM calls and connects LLM prompts,...</p>
</div>
<div class="proof-card">
  <span>E-action-taxonomy</span>
  <p>The paper separates external grounding actions from internal memory actions, and decomposes internal actions into retrieval, reasoning, and learning.</p>
</div>
<div class="proof-card">
  <span>E-decision-cycle</span>
  <p>The decision procedure repeatedly uses reasoning and retrieval to propose, evaluate, and select alternatives during planning, then executes a grounding or learning action and...</p>
</div>
  </div>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Claim map</p>
  <h2>What the review actually supports.</h2>
  <div class="claim-grid">
    <div class="claim-card">
  <span>C1 · paper-supported</span>
  <p>CoALA organizes language agents around three architectural dimensions: information storage, action space, and decision-making procedure.</p>
  <small>E-framework-definition</small>
</div>
<div class="claim-card">
  <span>C2 · paper-supported</span>
  <p>The framework separates working memory from long-term episodic, semantic, and procedural memory.</p>
  <small>E-memory-taxonomy</small>
</div>
<div class="claim-card">
  <span>C3 · paper-supported</span>
  <p>CoALA divides agent actions into external grounding actions and internal retrieval, reasoning, and learning actions.</p>
  <small>E-action-taxonomy</small>
</div>
<div class="claim-card">
  <span>C4 · paper-supported</span>
  <p>The top-level agent controller is modeled as a repeated decision cycle with planning and execution stages.</p>
  <small>E-decision-cycle</small>
</div>
  </div>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Agent infrastructure</p>
  <h2>What changes if you build systems.</h2>
  <div class="apple-card-grid apple-card-grid--two">
    <div class="apple-card"><p>Treat agent memory as typed runtime state, not only prompt text: working, episodic, semantic, and procedural stores have different persistence and...</p></div>
<div class="apple-card"><p>Separate internal actions from external side effects. Retrieval, reasoning, and learning should be inspectable runtime operations before a grounding...</p></div>
<div class="apple-card"><p>The decision loop is the agent's main procedure, so production runtimes need explicit scheduling, observation capture, action parsing, and replay...</p></div>
<div class="apple-card"><p>Procedural memory is broader than model weights: prompts, parsers, code skills, retrieval procedures, and learning policies can all become mutable...</p></div>
  </div>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Caveats</p>
  <h2>What this does not prove.</h2>
  <div class="apple-card-grid apple-card-grid--two">
    <div class="apple-card"><p>CoALA is primarily a conceptual framework and survey, not a new empirical agent with benchmark results.</p></div>
<div class="apple-card"><p>The framework does not by itself solve safety, permissions, persistence, or exactly-once execution for side-effecting agents.</p></div>
<div class="apple-card"><p>The paper's categories are intentionally broad, so implementations still need concrete interface contracts and failure handling.</p></div>
<div class="apple-card"><p>Autonomous updates to source code, retrieval, learning, or decision-making are flagged as risky and underexplored.</p></div>
  </div>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Run it</p>
  <h2>No PoC yet.</h2>
  <p class="deck-note">No PoC yet - contributions welcome. A useful local PoC would implement a tiny CoALA runtime skeleton with typed working, episodic, semantic, and procedural stores plus logged internal/external actions.</p>
  <div class="reference-list">
    <ul><li>Paper: <a href="https://arxiv.org/abs/2309.02427">Cognitive Architectures for Language Agents</a></li><li>Project/code: <a href="https://github.com/ysymyth/awesome-language-agents">https://github.com/ysymyth/awesome-language-agents</a></li><li>historical-foundation: <a href="https://arxiv.org/abs/2205.03854">Soar</a></li><li>classified-example: <a href="https://arxiv.org/abs/2210.03629">ReAct</a></li><li>classified-example: <a href="https://arxiv.org/abs/2305.16291">Voyager</a></li></ul>
  </div>
</div>

Note: This deck is synthesized from `data/reviews/coala-sumers-2023.json`. Update the review record, then run `bun run build`.