agent-readings

Start Here

CoALA

CoALA turns language agents into a cognitive-architecture design problem by separating memory modules, internal and external actions, and a looping decision procedure.

6Claims
6Evidence

Why It Matters

For agent infrastructure, CoALA names the runtime pieces that need explicit interfaces: working state, long-term stores, tool/environment grounding, retrieval, reasoning, learning, and the top-level decision loop.

Problem

  • Early language agents often placed an LLM directly in a feedback loop with an external environment.
  • Newer agents added reasoning, planning, reflection, retrieval, and memory, but papers used inconsistent names for similar mechanisms.
  • Traditional cognitive architectures had mature notions of working memory, long-term memory, grounding, learning, and decision cycles, but required hand-specified symbolic rules.

Key gap: The paper addresses the lack of a systematic framework for comparing existing language agents and planning new ones at the architectural level.

Method

Map language agents onto a cognitive architecture: keep active state in working memory, persist experience and knowledge in long-term memories, expose grounding/retrieval/reasoning/learning as actions, and run a repeated decision cycle that plans before executing environment or memory-changing actions.

  • Frame LLM calls and prompt chains as production-like operations over text.
  • Import cognitive-architecture concepts to define memory modules and action types.
  • Classify recent agents by their long-term memory, external grounding, internal actions, and decision-making procedure.
  • Use the classification to identify reusable abstractions and underexplored design directions.

Claims

C1 · paper-supported

CoALA organizes language agents around three architectural dimensions: information storage, action space, and decision-making procedure.

Scope: Framework definition in the introduction and Section 4.

  • E-framework-definition: The paper states that CoALA organizes agents by information storage, action space, and decision-making procedure, then develops those concepts in Section 4.

C2 · paper-supported

The framework separates working memory from long-term episodic, semantic, and procedural memory.

Scope: Memory taxonomy in Section 4.1.

  • E-memory-taxonomy: CoALA defines short-term working memory and long-term episodic, semantic, and procedural memories. Working memory persists active state across LLM calls and connects LLM prompts, long-term memory, and grounding interfaces.

C3 · paper-supported

CoALA divides agent actions into external grounding actions and internal retrieval, reasoning, and learning actions.

Scope: Action-space definition in Section 4 and Figure 5.

  • E-action-taxonomy: The paper separates external grounding actions from internal memory actions, and decomposes internal actions into retrieval, reasoning, and learning.

C4 · paper-supported

The top-level agent controller is modeled as a repeated decision cycle with planning and execution stages.

Scope: Decision-making procedure in Sections 4.6 and Figure 4B.

  • E-decision-cycle: The decision procedure repeatedly uses reasoning and retrieval to propose, evaluate, and select alternatives during planning, then executes a grounding or learning action and loops after observation.

C5 · paper-supported

CoALA can express a range of existing agents, including SayCan, ReAct, Voyager, Generative Agents, and Tree of Thoughts, by comparing their memory, grounding, internal actions, and decision-making.

Scope: Case studies and Table 2.

  • E-case-studies: Table 2 casts SayCan, ReAct, Voyager, Generative Agents, and Tree of Thoughts into the CoALA dimensions, showing differences in long-term memory, external grounding, internal actions, and decision-making.

C6 · paper-supported

The paper argues for modular, reusable language-agent abstractions such as Memory, Action, and Agent classes to reduce technical debt and improve compatibility.

Scope: Actionable insights in Section 6.

  • E-actionable-insights: The paper recommends systematic modular design, argues for standardized abstractions analogous to MDP/Gym concepts in reinforcement learning, and suggests company-wide agent libraries for reuse and testing.

Agent Infrastructure Takeaways

  • Treat agent memory as typed runtime state, not only prompt text: working, episodic, semantic, and procedural stores have different persistence and mutation semantics.
  • Separate internal actions from external side effects. Retrieval, reasoning, and learning should be inspectable runtime operations before a grounding action touches the world.
  • The decision loop is the agent's main procedure, so production runtimes need explicit scheduling, observation capture, action parsing, and replay boundaries.
  • Procedural memory is broader than model weights: prompts, parsers, code skills, retrieval procedures, and learning policies can all become mutable agent state.
  • The framework is useful as a schema for reviews because it forces each paper to say which memory, action, and decision components it actually changes.

Caveats

  • CoALA is primarily a conceptual framework and survey, not a new empirical agent with benchmark results.
  • The framework does not by itself solve safety, permissions, persistence, or exactly-once execution for side-effecting agents.
  • The paper's categories are intentionally broad, so implementations still need concrete interface contracts and failure handling.
  • Autonomous updates to source code, retrieval, learning, or decision-making are flagged as risky and underexplored.
  • The paper assumes text is the dominant internal representation, while multimodal and embodied settings may require richer state.