agent-readings · MemGPT

<div class="slide-frame slide-frame--center">
  <p class="deck-kicker">agent-readings / Memory and State</p>
  <h1>MemGPT</h1>
  <p class="deck-subtitle">Virtual memory for fixed-context agents</p>

<div class="deck-meta-strip">
    <div class="deck-meta-pill"><strong>2023</strong><span>arXiv 2023; revised 2024</span></div>
    <div class="deck-meta-pill"><strong>7</strong><span>claims</span></div>
    <div class="deck-meta-pill"><strong>7</strong><span>evidence refs</span></div>
    <div class="deck-meta-pill"><strong>paper-read</strong><span>status</span></div>
  </div>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">The one sentence</p>
  <h2>MemGPT treats the context window as RAM and external stores as disk, then gives the model functions for paging memory in and out.</h2>
  <p class="deck-note">The paper turns memory from a retrieval add-on into runtime state. For agent infrastructure, the key move is that memory operations become logged, inspectable actions with their own policy and failure modes.</p>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Before this paper</p>
  <h2>The friction was structural.</h2>
  <div class="apple-card-grid apple-card-grid--three">
    <div class="apple-card"><p>Fixed context windows limit long conversations and document analysis.</p></div>
<div class="apple-card"><p>Blindly extending context is expensive because attention cost grows quadratically and long-context models may still fail to use middle context reliably.</p></div>
<div class="apple-card"><p>Naive summarization loses details that a later agent step may need.</p></div>
  </div>
  <div class="deck-callout">
    <span>Key gap</span>
    <p>The paper asks how to make a fixed-context model behave as if it had larger, persistent memory without retraining the model or increasing the context window.</p>
  </div>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Core idea</p>
  <h2>Wrap a fixed-context LLM in an OS-like runtime that stores prompt-resident state in main context, out-of-context state in recall and archival storage, and...</h2>
  <p class="deck-note">Wrap a fixed-context LLM in an OS-like runtime that stores prompt-resident state in main context, out-of-context state in recall and archival storage, and exposes memory-management functions the model can call.</p>
  <div class="idea-loop" aria-label="Agent loop">
    <div><strong>Model</strong><span>reason</span></div>
    <div><strong>Runtime</strong><span>act</span></div>
    <div><strong>World</strong><span>observe</span></div>
  </div>
  <div class="step-strip">
    <div class="step-item">
  <span>01</span>
  <p>Store static instructions, writable working memory, and recent events in main context.</p>
</div>
<div class="step-item">
  <span>02</span>
  <p>Append user and model messages to recall storage so evicted history remains queryable.</p>
</div>
<div class="step-item">
  <span>03</span>
  <p>Store long documents or important facts in archival storage, implemented in experiments with PostgreSQL, pgvector, and HNSW indexing.</p>
</div>
<div class="step-item">
  <span>04</span>
  <p>Insert memory-pressure warnings before context overflow so the model can preserve useful information.</p>
</div>
  </div>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Action grammar</p>
  <h2>The agent is an interface contract.</h2>
  <div class="apple-card-grid apple-card-grid--two">
    <div class="apple-card"><p>Incoming event: user message, system message, timed event, document upload, or memory-pressure alert.</p></div>
<div class="apple-card"><p>Main context: system instructions, working context, and FIFO queue are concatenated into the prompt.</p></div>
<div class="apple-card"><p>Memory warning: queue manager warns the model before context overflow.</p></div>
<div class="apple-card"><p>Memory edit: the model writes durable facts to working context or archival storage.</p></div>
  </div>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Executable shape</p>
  <h2>The mechanism should fit on one screen.</h2>

```python
for event in event_stream:
    queue.append(render(event))
    recall_storage.write(event)

if prompt_tokens(queue, working_context) > warning_threshold:
        queue.append(system_memory_pressure_warning())

while true:
        output = llm(system_instructions, working_context, queue)
        if output.is_memory_function():
            result = execute_memory_function(output)
            queue.append(result)
            if output.request_heartbeat:
                continue
        else:
            recall_storage.write(output)
            send_to_user(output)
        break

if prompt_tokens(queue, working_context) > flush_threshold:
        evicted = queue.flush_old_messages()
        recall_storage.write(evicted)
        queue.summary = summarize(queue.summary, evicted)
```
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Evidence</p>
  <h2>The proof objects.</h2>
  <div class="proof-grid">
    <div class="proof-card">
  <span>E-memory-hierarchy</span>
  <p>The paper proposes virtual context management: main context holds prompt-resident state, external context holds out-of-context data, and the LLM uses functions to retrieve,...</p>
</div>
<div class="proof-card">
  <span>E-main-external-context</span>
  <p>Main context is split into system instructions, writable working context, and a FIFO queue. The queue manager writes messages to recall storage, warns near context overflow,...</p>
</div>
<div class="proof-card">
  <span>E-control-flow</span>
  <p>The function executor parses model outputs as memory function calls, feeds results and runtime errors back into context, and supports immediate follow-up inference through a...</p>
</div>
<div class="proof-card">
  <span>E-dmr</span>
  <p>DMR accuracy and ROUGE-L improve from GPT-3.5 Turbo 38.7% / 0.394 to MemGPT 66.9% / 0.629, GPT-4 32.1% / 0.296 to MemGPT 92.5% / 0.814, and GPT-4 Turbo 35.3% / 0.359 to MemGPT...</p>
</div>
  </div>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Claim map</p>
  <h2>What the review actually supports.</h2>
  <div class="claim-grid">
    <div class="claim-card">
  <span>C1 · paper-supported</span>
  <p>MemGPT frames the context window as main memory and external stores as disk-like memory, with the model responsible for moving information between tiers...</p>
  <small>E-memory-hierarchy</small>
</div>
<div class="claim-card">
  <span>C2 · paper-supported</span>
  <p>The runtime separates system instructions, writable working context, FIFO queue, recall storage, and archival storage, and uses memory-pressure warnings plus...</p>
  <small>E-main-external-context</small>
</div>
<div class="claim-card">
  <span>C3 · paper-supported</span>
  <p>MemGPT makes memory management event-driven and supports function chaining so the model can run multi-step memory searches before yielding control.</p>
  <small>E-control-flow</small>
</div>
<div class="claim-card">
  <span>C4 · paper-supported</span>
  <p>On deep memory retrieval over Multi-Session Chat, MemGPT substantially improves accuracy and ROUGE-L over fixed-context baselines for GPT-3.5, GPT-4, and...</p>
  <small>E-dmr</small>
</div>
  </div>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Agent infrastructure</p>
  <h2>What changes if you build systems.</h2>
  <div class="apple-card-grid apple-card-grid--two">
    <div class="apple-card"><p>A context window is main memory, not durable memory. Persistent agent state needs explicit storage tiers and movement rules.</p></div>
<div class="apple-card"><p>Memory actions should be first-class runtime events: search, write, flush, summarize, page, and heartbeat all need logs and policy.</p></div>
<div class="apple-card"><p>Memory pressure is a useful control signal. Long-running agents should know before context eviction happens, not after facts disappear.</p></div>
<div class="apple-card"><p>The memory manager is only as reliable as the base model's function-calling and stopping behavior.</p></div>
  </div>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Caveats</p>
  <h2>What this does not prove.</h2>
  <div class="apple-card-grid apple-card-grid--two">
    <div class="apple-card"><p>The paper's memory policy is prompt- and model-driven rather than formally specified or verified.</p></div>
<div class="apple-card"><p>Several evaluations rely on LLM judges or similarity metrics, not executable ground truth.</p></div>
<div class="apple-card"><p>The document-QA experiment uses a small sample of 50 questions.</p></div>
<div class="apple-card"><p>Performance depends strongly on model function-calling quality; GPT-3.5 is notably weaker in the paper's experiments.</p></div>
  </div>
</div>

---

<div class="slide-frame">
  <p class="deck-kicker">Run it</p>
  <h2>No PoC yet.</h2>
  <p class="deck-note">No PoC yet - contributions welcome. A compact PoC would wrap a small local LLM or mock policy with a FIFO queue, recall log, vector-like archival search, memory-pressure warnings, and heartbeat-based multi-step lookup.</p>
  <div class="reference-list">
    <ul><li>Paper: <a href="https://arxiv.org/abs/2310.08560">MemGPT: Towards LLMs as Operating Systems</a></li><li>Project/code: <a href="https://research.memgpt.ai">https://research.memgpt.ai</a></li><li>extends: <a href="https://arxiv.org/abs/2304.03442">Generative Agents</a></li><li>analogy: <a href="https://dl.acm.org/doi/10.1145/3600006.3613165">PagedAttention</a></li><li>builds-on: <a href="https://arxiv.org/abs/2210.03629">ReAct</a></li></ul>
  </div>
</div>

Note: This deck is synthesized from `data/reviews/memgpt-packer-2023.json`. Update the review record, then run `bun run build`.