The .txt File as the Soul of a Personal AI — FileRAG Memory Architecture

By Dharanidharan J (JD)

Full Stack & AI Engineer | Building Jarvix

The Problem Nobody Talks About

Every chatbot tutorial teaches you the same thing:

history = []
history.append({"role": "user", "content": message})

And that works — until it doesn't.

After 500 turns, your dict has forgotten who the user is. After 1000 turns, you're hitting token limits. After a restart, everything is gone. Redis helps with persistence but still buries early facts under noise. Vector DBs help with retrieval but bloat storage and need infrastructure.

What if the memory itself was just a file?

The Idea

Every conversation a user has gets distilled into a plain .txt file. That file is the brain. On every new query, a hybrid BM25 + semantic RAG retrieves the most relevant chunks from it and injects them as context.

users/
└── jd.txt        ← the soul file

The soul file looks like this:

[Turns 1-5]
- User's name is JD, software engineer
- Building FileRAG, a novel memory architecture
- Uses Pop!_OS with Fish shell and NVIDIA GPU

[Turns 6-10]
- Has a cat named Pixel who distracts during coding
- Paused TaskNest due to burnout
- Now focused on AgenticMesh

Human readable. Editable. Yours.

Why This Is Different

Most memory systems store messages. FileRAG stores a relationship.

System	What it stores
Dict / Redis	Raw message objects
Vector DB	Embeddings of messages
FileRAG	Distilled understanding of the user

The longer you use it, the more the AI understands you — not because it has more messages, but because it has a better summary of who you are.

The Architecture

User message
     ↓
Topic drift check (cosine similarity)
     ├── Drift detected → distill current buffer immediately
     └── No drift → continue
     ↓
Hybrid retrieval (BM25 + ChromaDB) from soul file
     ↓
Inject context → LLM responds
     ↓
Append to turn buffer
     ↓
Every 5 turns → distill → append to soul file → update ChromaDB
     ↓
Emergency distillation on exit (SIGINT/SIGTERM)

Key innovations:

1. Topic-Drift Distillation

Instead of waiting every N turns blindly, the system measures semantic similarity between the current buffer and the new message. If similarity drops below 0.25, it immediately distills and starts a fresh block. This keeps topic chunks clean and isolated.

2. Deduplication

Before writing any new chunk, cosine similarity is checked against all existing chunks. If >92% similar, the chunk is skipped. This prevents filler conversations from polluting the soul file.

3. Emergency Exit Handler

SIGINT and SIGTERM are intercepted. On Ctrl+C, the current buffer is immediately distilled before the process exits. Nothing is lost.

4. Hybrid Retrieval

BM25 catches exact keywords (project names, usernames). Semantic search catches meaning (preferences, personality). Together they outperform either alone.

Benchmark Results

Tested on Pop!_OS, NVIDIA GPU, sentence-transformers all-MiniLM-L6-v2 embeddings

Small (20 turns)

Metric	Dict	Redis	Vector DB	FileRAG
Write Speed (ms)	0.0004	0.30	33.38	20.33
Read Speed (ms)	0.002	0.26	6.64	9.26
Storage (KB)	1.42	1.38	396.16	356.66
Accuracy	100%	100%	67%	100%
Persistent	❌	✅	✅	✅

At small scale, Dict and Redis win on speed. FileRAG matches on accuracy. Fair.

Medium (500 turns)

Metric	Dict	Redis	Vector DB	FileRAG
Write Speed (ms)	0.0002	0.08	22.23	18.93
Read Speed (ms)	0.002	0.24	8.51	7.64
Storage (KB)	34.75	33.77	1604.16	653.47
Accuracy	0% ❌	0% ❌	67%	100%

This is where it gets interesting. Dict and Redis completely fail — core facts buried under 490 turns of noise. FileRAG still retrieves perfectly.

Large (1000 turns)

Metric	Dict	Redis	Vector DB	FileRAG
Storage (KB)	69.47	67.51	4338.36	938.74
Soul file only (KB)	—	—	—	18.58
Accuracy	0% ❌	0% ❌	67%	100%

FileRAG's total storage includes ChromaDB index overhead. The soul file itself — the actual human-readable memory — is just 18 KB for 1000 turns.

---|---|---|---|---|
| Storage (KB) | 3,478 | 3,381 | 76,159 | 29,812 |
| Soul file (KB) | — | — | — | ~1,865 |
| Accuracy | 0% | 0% | ~67% | ~100% |

At 100k turns, Vector DB would consume ~74 MB just for index storage. FileRAG's soul file stays under 2 MB — human-readable, portable, private.

The Verdict

Category	Dict	Redis	Vector DB	FileRAG
Fastest write	✅	✅	❌	Medium
Best accuracy at scale	❌	❌	Medium	✅
Smallest storage at scale	❌	❌	❌	✅
Persistent	❌	✅	✅	✅
No infrastructure	✅	❌	❌	✅
Local / offline	✅	❌	⚠️	✅
Privacy (on device)	✅	❌	⚠️	✅
Grows naturally with user	❌	❌	Medium	✅

FileRAG is not the fastest. It is not the simplest. But it is the only architecture that gets more accurate as the conversation grows, without growing infrastructure requirements.

The Human Analogy

Your brain doesn't record every conversation verbatim. It compresses experiences into memory — feelings, facts, patterns. The hippocampus distills, the cortex stores.

FileRAG does the same thing:

Conversation → Distillation → Soul file → Retrieval → Natural response
Experience  → Hippocampus  → Cortex    → Recall    → Natural behaviour

The soul file is not a database. It is a diary the AI reads before speaking to you.

Limitations (Honest)

Write speed is slower than Dict/Redis — embedding costs time
Not suited for 10k+ concurrent users — file I/O doesn't scale horizontally
Distillation quality depends on the LLM used — a weak summariser produces weak memories
10k+ turns require deduplication to prevent soul file pollution (implemented, but adds complexity)

Stack

LLM          → Groq (llama3-70b-8192)
Distillation → Groq (llama3-70b-8192) every 5 turns or on topic drift
Embeddings   → sentence-transformers/all-MiniLM-L6-v2
Vector store → ChromaDB (persistent)
Retrieval    → Hybrid BM25 + Cosine Semantic
Memory       → {user_id}.txt — the soul file