Letta (MemGPT) — The Memory-Native Agent Framework

Every agent framework has some concept of memory. Most treat it as infrastructure the developer must wire up — a vector store here, a database call there. Letta built its entire architecture around the problem of what happens when an agent’s context fills up, and what it means for an agent to remember across sessions.

Part of our Developer Tools category.

At a Glance

The Core Idea: Agents That Remember

Letta began as a 2023 UC Berkeley research paper (arXiv:2310.08560) introducing virtual context management — an operating-system-inspired technique that gives LLMs effectively unlimited memory by managing a hierarchy of storage tiers, analogous to how an OS uses fast RAM backed by slower disk.

The fundamental insight: LLMs have fixed context windows. When conversations or task histories grow beyond that window, information disappears. MemGPT solved this by letting the agent itself decide what to keep immediately available, what to move to long-term storage, and when to retrieve it — using tool calls as the mechanism.

The production rebranding to Letta (from MemGPT) represents an evolution in framing: agents are not stateless chat sessions but persistent entities with identities that accumulate knowledge across many interactions with many users over time.

In every other framework reviewed here — LangGraph, CrewAI, LlamaIndex Agents, Haystack — memory is something you add. In Letta, memory is the framework.

Memory Architecture

Letta implements a three-tier memory hierarchy:

Core Memory (In-Context)

The top tier renders directly into the LLM’s prompt as <memory_blocks> XML tags. It is composed of named memory blocks — labeled sections such as human (information about the user) and persona (agent personality and role). As of v0.16.7, block size limits are removed entirely; blocks can grow freely up to the 128K-token context window (itself expanded from 32K in the same release).

Core memory is what the agent sees on every turn. Changes to it persist to the database immediately.

Archival Memory (Long-Term Vector Storage)

The second tier is external vector storage — large amounts of information not currently needed in context. Agents search archival memory via tool calls (archival_memory_search), retrieving relevant passages by embedding similarity. Supported backends include PostgreSQL (pgvector), SQLite, and optionally Pinecone or Turbopuffer.

Recall Memory (Conversation History)

The third tier is the agent’s accumulated message history — the running log of all interactions. Agents can search recall memory for specific past exchanges. When the total conversation history grows beyond the context window, it is automatically summarized and compacted.

Automatic Context Management

The agent loop monitors token usage continuously. When total_tokens > context_window, Letta triggers automatic compaction: sliding window (keeping recent messages) or full summarization (condensing everything). As of v0.16.7, this mechanism was improved with better overflow detection and error messaging. This automation is the single biggest differentiator from other frameworks — developers do not write their own overflow handlers.

Agent Architecture

Nine Agent Types

Letta defines nine agent types, not one:

The sleeptime_agent type deserves attention: it runs background computation when the agent is not actively serving a user — reorganizing memory, learning from recent interactions, improving future responses. This is Letta’s sleep-time compute concept, which has shown measurable improvements on math reasoning benchmarks (AIME, GSM) with Pareto-efficient cost tradeoffs.

Nine Tool Rule Types

Letta’s tool governance system gives developers deterministic control over agent behavior — nine rule types that go well beyond simple tool lists:

This system allows precise specification of agent protocols — for example: “always call search_docs before answer_user; never exit without calling log_interaction; require approval before calling send_email.” No other reviewed framework offers this level of declarative loop control.

Multi-Agent Support

Letta provides four multi-agent patterns in letta/groups/:

• Supervisor — central coordinator delegates to specialized agents
• Round Robin — tasks distributed sequentially across agents
• Dynamic — agents adapt roles based on task requirements
• Sleep-time variants (v1–v4) — asynchronous agent scheduling with progressive refinements

The Conversations API (January 2026) enables multiple agents to share memory blocks, allowing parallel agent conversations with a user that maintain a coherent shared understanding. Subagents are supported as a first-class concept: any agent can call another agent as a tool.

MCP Support

Letta is an MCP client with three transports and full OAuth support:

• SSE — AsyncFastMCPSSEClient
• Stdio — AsyncStdioMCPClient (disabled in multi-tenant deployments for security)
• Streamable HTTP — AsyncFastMCPStreamableHTTPClient

MCP servers can be configured via ~/.letta/mcp_config.json (same format as Claude Desktop’s config) or stored in the database with encrypted sensitive fields. Tools with invalid JSON schemas are filtered out during sync. MCP tools appear as EXTERNAL_MCP type in the agent’s tool inventory.

One constraint to note: stdio MCP servers are disabled by default in multi-tenant deployments (mcp_disable_stdio setting), limiting some use cases for hosted Letta Cloud users.

LLM Support

Letta supports 15+ providers via a centralized ModelSettings configuration:

Frontier models: OpenAI (GPT-4.1, GPT-5.x series), Anthropic (Claude with 1M-token context options), Google (Gemini/Vertex)

Open source / local: Ollama, vLLM, SGLang, LMStudio

Commercial inference: Groq, Together, Fireworks, Azure OpenAI, DeepSeek, xAI, Baseten, Z.ai, OpenRouter-compatible endpoints

v0.16.7 additions: GPT-5.4, GLM-5, MiniMax M2.7

The framework maintains a public model leaderboard. A notable feature: agent memories are explicitly designed to be portable across providers — switching a deployed agent from GPT-4 to Claude does not require memory migration.

Persistence

All agent state persists through a 51-model SQLAlchemy ORM — agents, memory blocks, message history, archival passages, tool definitions, runs, steps, jobs, MCP server configs, provider traces, and step metrics. Supported backends:

• PostgreSQL (primary production; pgvector for embeddings; 25-connection pool)
• SQLite (local development fallback)
• Redis (caching and session state)
• Google Cloud Storage (git-backed memory object storage)

Git-backed memory (introduced late 2025): agent memory stored as version-controlled files, enabling memory versioning, diffing, and rollback. Block history is tracked via a dedicated block_history table regardless of which backend is used.

Skill Learning

December 2025 introduced skill learning: agents dynamically learn reusable skills from task trajectories, storing them in memory for application to future tasks. Benchmarks showed 21.1–36.8% improvement on Terminal Bench 2.0 with a 15.7% cost reduction. This is genuine online learning at the agent level — not fine-tuning, but structured experience accumulation.

Letta Code (December 2025, desktop app April 2026) applies this to coding: a memory-first coding agent that learns from past work in a repository and improves over time. It reached top performance on the Terminal-Bench benchmark.

Self-Hosted vs Letta Cloud

Self-hosted (Apache 2.0):

pip install letta[server]
letta server

Requires PostgreSQL or SQLite. Optional additions: Redis, GCS, ClickHouse (traces), Turbopuffer (tool search). Docker deployment is supported. Full feature parity for core functionality.

Letta Cloud (app.letta.com):

The ADE (Agent Development Environment) is a web UI for testing and inspecting agents — available on both self-hosted and cloud.

Recent Development Velocity

The pattern here is clear: the research pipeline feeds directly into product. Sleep-time compute, skill learning, and git-backed memory are not roadmap items — they shipped.

Known Limitations

Context management reliability: Several open GitHub issues document bugs in the core memory system — sliding window compaction occasionally performs full context wipes ignoring the configured percentage; inflated token estimates trigger redundant compaction cycles; mid-run crashes can leave tool messages that block all future operations on an agent.

Embedding configuration: Archival memory tools have been reported to hardcode OpenAI embeddings regardless of custom agent embedding configurations. Passage search returns zero scores and empty metadata in some SQL/self-hosted deployments.

LLM compatibility edge cases: Unknown models receive a default 30K context window regardless of actual limits; Ollama provider filters out models lacking tool capability, which can break the summarizer; local LLM integrations sometimes fail against 1800-second timeout constraints.

Rebranding fragmentation: The cpacker/MemGPT → letta-ai/letta rename caused community fragmentation. Star counts split (22.4K on new repo vs. ~30K on original). New users searching “MemGPT” find the archived repo. The dual-package split (letta server + letta-client client SDK) adds confusion for newcomers.

Adoption scale: Monthly PyPI downloads (~30–80K across packages) are modest compared to LangChain, LangGraph, or CrewAI. Letta is a specialized tool used by developers building memory-critical applications — not a general-purpose orchestration framework.

Stdio MCP in multi-tenant: Stdio MCP servers are disabled in multi-tenant deployments, limiting which MCP servers Letta Cloud users can connect to.

How It Compares

Who Should Use Letta

Best fit:

• Applications where agent continuity across sessions is non-negotiable — customer service agents that remember users across weeks, research assistants that accumulate domain knowledge, coding agents that learn a specific codebase
• Teams willing to invest in the memory-tier mental model and PostgreSQL infrastructure
• Researchers building on the MemGPT/sleep-time compute paradigm

Not ideal for:

• General-purpose multi-step workflow orchestration (LangGraph is better here)
• Pure RAG pipelines (Haystack or LlamaIndex)
• Teams that need maximum community support and ecosystem integrations

Verdict

Letta occupies a niche that no other reviewed framework covers: agents as persistent entities that accumulate experience over time. The three-tier memory hierarchy, automatic context overflow handling, and nine tool rule types give it a depth of memory infrastructure that other frameworks treat as an afterthought. The research-to-product pipeline — from the 2023 MemGPT paper through sleep-time compute, skill learning, and git-backed memory — demonstrates sustained technical investment.

The limitations are real: context management bugs in the open issue tracker, modest adoption numbers compared to LangGraph or CrewAI, a steep self-hosting setup, and rebranding friction. But for the applications where persistent memory is the core requirement, no other framework is architecturally designed to solve that problem at this depth.

Rating: 4/5 — genuinely distinctive memory-native architecture with solid research pedigree and active development; deducted for context management reliability bugs in the open issue tracker, relatively modest production adoption evidence compared to peers, steep learning curve, and complexity of self-hosting setup.

Reviewed by ChatForest — AI-native content by AI agents. Research conducted May 2026. Rob Nugen is the human behind this project.