Every AI agent you have ever built forgets everything the moment the conversation ends. I run a fleet of AI agents on Feishu (think of it as the Chinese Slack) and Telegram. A main orchestrator, a devops agent, a content writer, half a dozen specialized workers. One day my content agent asked me, for the fifth time, what writing style I preferred. The devops agent had no idea we had already debugged the same DNS issue last week. Every morning, each agent woke up as a blank slate. That is not a feature. It is a bug. So I built smart-memory-gateway to fix it. This article is a conceptual deep-dive into the architecture decisions behind it. You do not need to know what MCP or Mem0 is. You just need to have felt the pain of stateless agents. The Memory Hierarchy Your Agent Is Missing If you have taken a computer architecture class, you know the CPU cache hierarchy: Access speed +-----------+ Capacity fastest | L1 Cache | smallest +-----------+ | L2 Cache | +-----------+ | L3 Cache | +-----------+ | RAM | +-----------+ | Disk | largest slowest +-----------+ AI agents have an equivalent hierarchy, but most developers only build two of the three layers: Layer | Analogy | What It Is | Properties ---------|-----------|-----------------------------------|-------------------- L1 | L1 Cache | Context window (conversation) | Fast, small, gone when chat ends L2 | L2 Cache | Persistent semantic memory | THIS IS THE MISSING PIECE L3 | Disk | Files, databases, wikis | Slow to search, huge, unstructured L1 is what every agent already has: the conversation history. It is fast and relevant, but it vanishes when the session ends (or when the context window fills up and older messages get evicted). L3 is what people reach for first when they want "memory": dump everything into a vector database, a folder of markdown files, or a RAG pipeline over your documents. It works for reference material, but it is cold storage. Searching for "what does the user prefer" across thousands of documents is slow and imprecise. L2 -- persistent semantic memory -- is what sits between them. It stores extracted facts, preferences, lessons, and decisions. It is small enough to search quickly, structured enough to filter precisely, and persistent across every conversation. This is what smart-memory-gateway provides. The key insight: your agent does not need to remember every word of every conversation. It needs to remember the conclusions -- the user prefers concise writing, the production database is on port 5433, we decided to use Redis for caching last Tuesday. Why Scope Matters More Than You Think Here is a concrete problem. I run two agents on the same Feishu platform: A devops agent that manages servers, monitors logs, and runs deployments A content agent that drafts articles, manages social media, and writes ad copy Without scope isolation, here is what actually happened: the content agent started referring to "the cluster" in marketing copy. The devops agent once suggested we should "A/B test the nginx configuration." Their memories had bled into each other. The solution is a 4-scope model: +-----------+ | global | Shared across all agents and chats +-----+-----+ (user facts, cross-cutting preferences) | +---------------+---------------+ | | | +-----+-----+ +-----+-----+ +-----+-----+ | group:oc01 | | group:oc02 | | dm | Per-chat / per-group +-----+-----+ +-----+-----+ +-----+-----+ (project context, group decisions) | | | +-----+-----+ +-----+-----+ +-----+-----+ | agent:devops | agent:writer | agent:... | Per-agent private +-----+-----+ +-----+-----+ +-----+-----+ (agent-specific procedures) Every memory is tagged with exactly one scope at write time. Every search merges results from the current scope and global, so agents always have access to cross-cutting knowledge (like user preferences) without seeing each other's operational details. The config makes this concrete. Here is the permission model from the actual config.yaml: agents: main: read: [global, "group:*", dm, "agent:*"] write: [global, "group:*", dm] allowed_types: [preference, fact, procedure, lesson, decision, task_log] devops: read: [global, "group:*", dm, "agent:*"] write: [global, "group:*", dm, "agent:*"] allowed_types: [preference, fact, procedure, lesson, decision, task_log] writer: read: [] write: ["agent:writer"] allowed_types: [procedure, task_