A chatbot can sound convincing for five minutes without remembering anything.
Then you mention the job interview you were stressed about last week, the name of your dog, or a small detail from a late-night conversation. It replies like none of it happened.
That is where most "AI companion" demos fall apart.
I am building Local Waifu, a desktop AI companion that runs on the user's own Mac or PC. One of the rules I set early was simple: conversations and memories should stay on the machine. No central chat database. No server that needs to be online for the character to remember someone.
The rule sounds clean. Building it was not.
Saving chats is not memory
The first version of memory was the obvious one: save messages.
That gives you history, which is useful, but it does not solve recall. A long chat history grows fast. Sending all of it back to a local language model on every message is slow, expensive in context space, and usually makes the reply worse.
The model does not need to see every conversation from the last six months.
It needs the few pieces that matter right now.
If someone says, "I have to take Luna to the vet tomorrow," the character should be able to find that Luna is their dog. It should not need to reread hundreds of unrelated messages about work, movies, and dinner plans to get there.
So I treated chat history and long-term memory as different things.
Chat history is the recent conversation. It gives the model immediate context.
Long-term memory is a small collection of facts, moments, preferences, and relationship details that may matter later. Those memories need to be searchable by meaning, not only by exact words.
The memory data stays in SQLite
I wanted the app to work without a hosted database, so the storage layer is local SQLite.
Each character gets their own data. Chats, memories, extracted entities, and relationships are stored locally on the device. If a user creates two characters, one character does not quietly inherit the other one's memories.
That separation matters more than it sounds.
A companion app is personal by design. Mixing context across characters is not a harmless bug. If one character starts talking about something belonging to another, the whole illusion disappears immediately.
For each memory, I store the text itself along with metadata such as:
- the character it belongs to
- when it was created
- importance
- emotional weight
- how often it has been used
- a vector embedding used for semantic search
The embedding is the part that lets the app search by meaning.
"Vet appointment" and "Luna is sick" may share no exact keyword. A semantic search can still see that they are probably connected.
I use 768-dimensional vectors stored directly in SQLite as binary data. At recall time, the app calculates similarity locally and ranks the results. No message content needs to leave the user's computer for that lookup.
Recall needs more than similarity
A pure similarity score is not enough.
Imagine a user mentioned their favorite game once two years ago, then spent the last month talking about a difficult family situation. Both memories might be related to a new message in some vague way. The more recent and more emotionally important memory should usually win.
So recall is weighted by more than vector similarity.
Importance matters. Recency matters. Emotional weight matters. A memory that has been useful before gets a little extra credit too.
There is no perfect formula here. I do not think there ever will be one.
A system that aggressively recalls every detail feels creepy and repetitive. A system that barely recalls anything feels empty. The work is mostly tuning that middle ground and accepting that a companion should sometimes not bring something up.
The user should feel remembered, not monitored.
I also extract a small knowledge graph
Some information is easier to retrieve as a relationship than as a paragraph of chat text.
For example:
- Luna is the user's dog
- Alex is the user's brother
- The user works night shifts
- The user dislikes phone calls
- A character enjoys rainy evenings
Those can be represented as entities and relationships.
Every few messages, the app can extract useful entities and links from the conversation. This happens locally through the model already running on the user's machine. The resulting graph is stored beside the other character data.
The graph is not there to replace memory. It fills a different role.
A vector memory is good at finding a moment with emotional context. A graph is good at answering structural questions such as "Who is Alex?" or "What is connected to this person?"
Both are useful. Neither should be trusted blindly.
Language models are very good at sounding confident while getting a relationship wrong. The extraction step needs validation, deduplication, and a way for the user to inspect or remove what was stored. A memory system without an escape hatch becomes frustrating fast.
The bug that made her forget everything
The most painful issue I found was not a database bug.
The app had the chat data. It had stored memory records. It had the retrieval logic.
But on some fresh installations, the local model used to create embeddings had not been downloaded yet. The app did not explain this clearly enough. Semantic recall returned no results, so the character behaved as if she had no memory of previous conversations.
Nothing crashed.
That made it worse.
A crash tells you there is a problem. A companion quietly forgetting what you told her last night can look like a limitation of the product itself.
The fix was straightforward after I found the cause:
- Ensure the embedding model is pulled during setup or startup.
- Retry the operation on the next launch if the model is not ready yet.
- Make the missing-model state visible instead of silently returning zero memories.
- Avoid falling back to reading the whole chat history every time, because that creates a different performance problem.
The lesson was simple: in AI products, a missing dependency can look exactly like bad intelligence.
Multilingual memory makes this harder
Local Waifu supports several languages, including Polish, Japanese, Korean, and Chinese.
Plain text search handles this badly. Even in English, people rarely repeat the same sentence. In different languages, the problem gets larger. A user can discuss the same person with nicknames, grammar changes, borrowed words, or a mix of two languages in one chat.
That is why semantic recall matters.
It is not magic. It will still make mistakes. But it gives the app a chance to connect related ideas instead of searching only for matching strings
