Batching interactions into typed memory records slashes construction token usage by more than 80% while nudging inference speed upward.
Before LycheeMemory V2, long‑horizon agents relied on eager turn‑level consolidation—each user turn triggered an LLM call to extract or summarize a new memory record. Systems such as A‑Mem therefore accrued huge token budgets as conversations grew, and coarse summarization risked losing fine‑grained evidence [1].
LycheeMemory V2 cuts construction tokens by 86.0% on the LoCoMo benchmark, using only 204.1 K tokens instead of A‑Mem’s 1.46 M. “On LoCoMo, LycheeMemory uses only 204.1K construction tokens, 86.0% and 86.6% lower than A‑Mem’s 1459.9K and Mem0’s 1520.8K respectively, and 58.3% lower than TiMem’s 489.5K” [1].
Query‑time token consumption also drops, with LycheeMemory needing just 4.01 K tokens—a 27.9% reduction versus A‑Mem. “On LoCoMo, LycheeMemory uses 4.01K query tokens, lower than A‑Mem’s 5.56K (-27.9%) and TiMem’s 10.71K (-62.6%)” [1].
Accuracy remains high, reaching 89.22% on LoCoMo and 92.20% on LongMemEval‑S, matching state‑of‑the‑art despite the token cuts [1].
The paper’s evaluation is limited to two synthetic benchmarks and a single LLM (GPT‑4.1‑Mini), leaving open how segment detection fares on noisy real‑world dialogues or with larger models; it also reports only modest throughput gains, so the claimed 2× speedup may not materialize in latency‑sensitive deployments.
If these numbers hold across domains, developers should replace turn‑level memory pipelines with semantic segment batching and typed record indexes as a default—re‑run LoCoMo or your own long‑context workloads to verify that construction costs halve without sacrificing answer quality. Will the next generation of LLM agents treat memory construction as a batch job rather than an after‑thought?
