Hands-on with Tencent Cloud Agent Memory: Giving AI a Real Long-Term Memory

I've been following an open-source project lately: Tencent Cloud's TencentDB Agent Memory (repo TencentCloud/tencentdb-agent-memory). The problem it tackles is simple but, until now, nobody had turned it into an engineering-grade system — how to stop an AI Agent from starting from scratch every single time.

This article is my learning notes written while reading the source code — not a sponsored post. I'll walk through its positioning, architecture, how memory "grows" layer by layer, how it retrieves and governs memory, and how to run it locally. At the end, I'll share why, as someone who works on both embedded systems and Agents, this project caught my attention.

1. What problem does it actually solve?

Anyone using Agents to write code or do ops has hit this: for the same project, you have to re-explain the context every time the conversation resets; the same document gets re-read by every Agent; a pitfall one workflow already hit gets hit all over again by the next Agent.

Re-explaining context, re-reading docs, re-discovering workflows — these three kinds of "repetition" are exactly what Agent Memory wants to eliminate.

Its own positioning, verbatim: team-level memory hub for AI Agents — turning conversations, documents, and code into four reusable memory assets (Chat Memory / Skill / LLM-Wiki / Code-Graph), then doing governance, sharing, and equipping between Agents and frameworks.

One distinction the project keeps stressing: it is not a chat-log warehouse. RAG only answers "can we find it"; Team Memory also has to answer "who can use it, which version is valid, which Agent should get it." The former is retrieval; the latter is governance.

Its slogan is blunt: Agents remember. Humans innovate.

2. How the four modules break down

The repo splits into four pieces with clear responsibilities:

An easy-to-confuse point: MemoryCore does not store knowledge content. For example, when you import a product doc, MemoryCore only registers "which knowledge source, what type, status, relationships, service address" — the actual parsing and retrieval are delegated to MemoryKnowledge. This "metadata/content separation" makes permissions and migration much easier later.

3. How memory "grows": L0–L3

This is the part I most wanted to understand. It layers memory into four levels: conversations first land in L0, then an async pipeline refines them into higher-level assets:

• L0 Conversation: Raw conversations with full context, timestamps, and source. Used to verify original statements.
• L1 Atom: Facts, preferences, constraints, and events extracted from conversations. Precise recall of actionable info.
• L2 Scenario: Knowledge blocks organized around a project/scenario. Quickly restore the working environment.
• L3 Core / Persona: Long-term profile, stable patterns, high-level cognition. Lets an Agent enter the user/team context from the start.

Refinement is asynchronous — not real-time distillation every turn, but a pipeline that slowly aggregates L0 upward in the background. Documents and code take a different path: documents → Wiki pages (with a link graph you can drill into); code → CodeGraph (indexing symbols, files, call relationships, impact paths).

The closed loop is the essence of this design. There's a line from the project I copied into my notes: "A loop without memory just repeats faster; a loop that inherits memory can beat the last iteration every time." Valuable interactions are stored as Chat Memory → validated workflows are distilled into Skills → doc/code changes trigger Wiki ingest and CodeGraph sync.

4. Retrieval & governance: doing one more step beyond "can be found"

On retrieval it's not picky: normally bootstrap quickly with L2/L3, fall back to L1/L0 when you need specific facts. Underneath is BM25 + vector retrieval + RRF (Reciprocal Rank Fusion). Results are also capped by three limits — item count, character budget, and timeout — to prevent the context from being stuffed in one shot. This detail is pragmatic; RAG usually fails by cramming too much into the prompt at once.

Governance is where it pulls ahead of a plain vector DB:

• Visibility semantics: private (owner only), team (team-readable), restricted (precise ACL by User/Role/Agent), agent (targeted equipping within the team).
• Fixed Binding + ACL: first narrow asset permissions by Team/User/Agent/visibility, then retrieve by query. Which assets a given Agent can actually use is decided by this mechanism.
• Generation provenance: the Prompt ID, version, source, and content SHA-256 actually used by L1/L2/L3 are all recorded, and you can pinpoint the generation log by Memory ID. But it doesn't store a snapshot of the Prompt body — guaranteeing traceability while avoiding hoarding the policy itself as data.

5. Cold start & team playbook

"Cold-start friendly" isn't a slogan. It can import an existing codebase (CodeGraph auto-indexes), documents (Wiki auto-generates), and historical sessions (auto-extracts Skills and Chat Memory) — the project calls this "loading the save file." A newly onboarded Agent team doesn't start from blank; it inherits existing experience.

The team-playbook example is also down-to-earth: a small team called "Tiny but Serious Inc." with members You / Scout / Builder / Reviewer, plus Agent Memory. Different roles equip different assets — Scout carries Chat Memory from user interviews and the market Wiki; Builder carries the product Wiki and CodeGraph. Same memory substrate, sliced into different views by role.

6. How to run it (local hands-on)

I followed along on the feat/server_team branch; spinning up MemoryCore locally is lightweight:

# MemoryCore is open-sourced as a Standalone Runtime
cd MemoryCore
npm install
npm run build

export TDAI_LLM_API_KEY="your-api-key"
export TDAI_LLM_BASE_URL="https://api.openai.com/v1"
export TDAI_LLM_MODEL="gpt-4o-mini"
node --import tsx src/gateway/server.ts

The Gateway listens on 127.0.0.1:8420 by default, using SQLite, local files, and in-process state. Other than the LLM API, there are no required external services, and it disables remote Embedding by default, using BM25 — very friendly for a local single machine. Data is written to ~/.memory-tencentdb/memory-tdai by default.

To launch the full stack in one go (memory-core + memory-hub + proxy), use the deploy script in the repo:

cd deploy/global-images
cp .env.example .env      # fill in two sets of LLM params (memory group + proxy group)
./start-all.sh            # prints a one-liner you can paste into Claude/CodeBuddy

The panel is at http://localhost:8125.

The Agent Adapter only needs to do three things, and the docs are refreshingly restrained: write L0 when the session ends or after each turn; recall L1/L2/L3 before constructing the Prompt; inject the recalled results into the Agent as bounded, identifiable context. For OpenClaw there's a ready-made openclaw-plugin/, for Hermes a hermes-plugin/, and for custom Runtimes the TS/Python SDK under sdk/memory-core/. The adapted Agent list includes DeepSeek Harness, Claude Code, Codex, CodeBuddy, WorkBuddy, Hermes, OpenClaw, with a Generic integration guide for anything not listed.

7. A few pitfalls and judgments I noted

A few points I hit (or predicted) while reading:

1. Wiki / CodeGraph are built asynchronously — after import you must wait for them to be ready; they're not queryable immediately. Any automation pipeline must budget for this latency.
2. CodeGraph currently prioritizes public HTTPS repos; private / SSH support is still maturing. On-prem codebases will have to wait.
3. v2 → v3 data migration must run a script first, and the docs explicitly say "back up the entire data directory before migrating." The data format goes from v2 to v3; before launching the new Gateway, probe with python scripts/migrate-v2-to-v3/v2-to-v3-migrate.py <data-dir> --dry-run.
4. On security: listening on a non-loopback address requires TDAI_GATEWAY_API_KEY; CORS is off by default — don't use *; all Secrets go through environment variables. These are the baseline.
5. There's a benchmark figure that speaks to the value: on PersonaMem, a relative improvement of +59% (48% → 76%) — long-term persona memory makes cross-session understanding of Agents markedly better.

Why am I watching this project? Half my energy goes into embedded systems like ESP32, the other half into Agents. On the embedded side I'm researching ESP-Claw — a framework that burns an Agent Runtime into silicon so devices can decide locally. Cloud Agents and edge Agents look like two ends, but the memory substrate can be the same: during development, use an Agent to write firmware and store the pitfalls you hit as Skills; once the on-device Agent connects to sensors via MCP, it can also return to the central memory for context. TencentDB Agent Memory's idea of "memory/execution separation, governance across frameworks" is exactly the missing piece in that picture.

Conclusion

In the Agent industry, the first two years were about "who has the stronger model, who prompts better." Now it's slowly shifting to "who remembers" — not remembered in the chat box, but remembered as governable, shareable, cross-framework-reusable assets.

Tencent Cloud's open-source project has engineered this: four layers of memory, four kinds of assets, a set of governance semantics, and a proxy layer with a stable protocol. Agents remember. Humans innovate. — now I have a bit of a feel for that line.

If you're also working on Agent memory, or have already integrated this system, drop a comment on the blog about your real-world experience — what's good, what's still awkward — that's worth more than the docs.

原文发表于 沐沐ai专题

关注「沐沐ai专题」公众号，获取更多 AI 实战干货