Someone in a thread says attention is just a Hopfield network, and the next reply runs with it: so the model already has associative memory, so an agent does not need anything else to remember. Both sentences sound like the same claim. Only the first one is true, and only in a narrow sense that is worth stating exactly, because the narrow version is useful and the loose version leads people to wrong architecture decisions.
One line to carry: one update step of a modern continuous Hopfield network equals scaled dot-product attention under a specific identification of queries, keys and values; that is an identity of operation, not a history of how transformers were built, and it does not turn a context window into memory that outlives the conversation.
The identity, stated precisely
The result is from Ramsauer and colleagues, in a 2020 paper titled Hopfield Networks is All You Need that appeared at ICLR 2021. The abstract states it in one sentence:
The new update rule is equivalent to the attention mechanism used in transformers.
The update rule of their modern Hopfield network is ξ_new = X softmax(β Xᵀξ). Read it as: compare the current state with every stored pattern, turn the similarities into softmax weights, return the weighted sum of patterns. Now identify the stored patterns with keys, the state with a query, add the value projection, and set β = 1/√d_k. The update becomes softmax(QKᵀ/√d_k)V, which is scaled dot-product attention.
Under those substitutions the equality is exact, at the level of that one operation. A query attends over keys and returns a weighted blend of values. A Hopfield state updates by attending over stored patterns and returning their weighted blend.
What the identity does not say
It is one update step. The paper's own text says that "retrieval with one update is compatible with activating the layers of deep networks", and its theorems bound how close a single update gets to the stored pattern when patterns are well separated. It is a statement about a step, not about a network iterating to convergence.
It is not a lineage. Attention was introduced in Attention Is All You Need in 2017, three years before the equivalence was shown. The paper's first figure uses an equality sign and explains it in the caption: the sign means "keeps the properties". The two were found to be the same operation after the fact.
It covers the operation and nothing around it. Residual connections, layer normalisation, the feed-forward blocks, causal masking and training dynamics are outside it. Later work narrows the conditions under which the identity holds rather than overturning it.
The title itself is a nod to the 2017 paper, and it belongs to a genre large enough to have been measured: a December 2025 preprint, All You Need is Not All You Need for a Paper Title, counts 717 arXiv titles containing the phrase between 2009 and 2025, 200 of them in 2025 alone, and argues the format favours "memorability over precision". This is one of the rare cases where the memorable title is also the theorem.
Three capacity numbers that get merged into one
Ask how many patterns a Hopfield network can store and you will see three different answers quoted as if they were the same. They answer three different questions.
| source | capacity | what it tolerates |
|---|---|---|
| Hopfield, 1982, simulations at N = 30 and 100 | about 0.15N | until recall errors become severe |
| Amit, Gutfreund and Sompolinsky, 1985 and 1987 | αc ≈ 0.138, often rounded to 0.14, so about 0.138N | a small fraction of bit errors, under 1.5 % at zero temperature |
| McEliece, Posner, Rodemich and Venkatesh, 1987 | n/(2 ln n) | none: exact recall of most memories |
There is no contradiction between them. The first is an empirical rule of thumb from small simulations. The second is a statistical mechanics result that allows a little noise in what comes back. The third demands exact recovery, so it is smaller and grows more slowly. The modern variants changed the question altogether: dense associative memories raised capacity to polynomial and then exponential scaling, and Ramsauer and colleagues prove exponential capacity in the dimension of the space for continuous states. If a post quotes 0.14N as the capacity of the network inside a transformer, it has taken the number from the wrong row of a table that stopped at 1987.
A vocabulary for what retrieval returns
The most practical part of the paper is not the equivalence but the taxonomy of fixed points it gives for the update. A query can settle on a single stored pattern, the clean case, when one pattern is close and well separated. It can settle on a metastable state that averages a subset of similar patterns. Or, when nothing is well separated, it can settle on a global average of everything stored.
The authors then used that lens on trained models. The abstract reports that attention heads "perform in the first layers preferably global averaging and in higher layers partial averaging via metastable states". A blend, in this reading, is an operating regime of the retrieval step and not automatically a malfunction: when the stored patterns are not separated enough, the step returns an average of a subset rather than one of them. That is a statement about one retrieval step. It is useful vocabulary, and it is not an explanation of every wrong answer a model gives.
What this means for an agent's memory
It gives a clean boundary. Attention retrieves associatively over what is in the context right now. It does that very well, and the Hopfield reading explains why. It does not keep anything once the context is gone.
Google Research's Titans paper draws the same line in its abstract:
From a memory perspective, we argue that attention due to its limited context but accurate dependency modeling performs as a short-term memory, while neural memory due to its ability to memorize the data, acts as a long-term, more persistent, memory.
So the reply in the opening thread has the direction backwards. Attention being a Hopfield update is a statement about how a model retrieves inside a window. What an agent needs to have in the window next week, after a restart, in a different session or a different tool, has to be stored somewhere that survives the session and put back in front of the model when it matters. The identity tells you what the retrieval step does with whatever gets there. It says nothing about getting it there.
Where did you last see the capacity of a Hopfield network quoted, and which of the three rows was it actually from?
Disclosure: I work on Mnemoverse, a memory service for AI agents, which is the kind of system the last section is about, so weigh that section accordingly. The longer version, with every paper and the full capacity table, is on our library, and the MCP server is open source (MIT): github.com/mnemoverse/mcp-memory-server.
