A 2.78-trillion-parameter model answered a prompt on one CPU inside 8.24 GB of resident memory, and the project reports that the same model on a 224 GB machine produces byte-identical output. Extra RAM buys clock time and nothing else. That inversion is the entire argument of kimi-k3-in-c, a Kimi K3 inference engine written in portable C99 with no BLAS, no framework and no GPU.
Memory becomes a dial instead of a floor
Most inference stacks treat memory as an eligibility test. You either hold the weights or you do not run. The README replaces that test with a ladder, measured on one machine with 124 cores and a fast NVMe drive: 26.5 seconds per token at 8 GB, 24.2 at 32 GB, 19.8 at 64 GB, and 5.6 at 128 GB and above. The first three rows still read the model off disk on every step, so a slower drive drags them down. The top row keeps everything resident and stops waiting on storage.
The layout behind that ladder is described in four parts. The dense trunk stays in memory to whatever depth you pick and the rest streams. The 1.45 TB of routed experts are never resident at all, and get multiplied straight out of their packed 4-bit form rather than being unpacked into floats first. The other reductions are architectural: KDA, described as attention with a memory that never grows, and MLA, one latent in place of ninety-six heads. On top of that sit routing choices you can inspect, including picking 16 experts out of 896 and packing 93 layers so each one costs a single read.
The expert cache is sized from evidence rather than intuition. A trace of 100,096 expert requests recorded during a full 93-layer run ships with the repository, and a Python script replays it to reprint the capacity table the docs publish.
The gate moved from VRAM to the disk
Nothing about the CPU requirement is exotic. AVX2 with FMA is enough and the README says AVX-512 is unnecessary. The build is seven C files against libm and OpenMP, and the resulting engine is 176 KB. Linux x86-64 is assumed because the code uses O_DIRECT, posix_memalign and getrusage.
The hard requirement is storage. The checkpoint is 1.56 TB, the packed trunk adds 109 GB, and the project asks for roughly 1.7 TB free on fast local disk. So the barrier did not disappear; it relocated. Anyone with a spare terabyte and a weekend of bandwidth is now in the same position as someone with a rack.
Verification you can run before you commit the download
The part worth copying has nothing to do with trillion-parameter models. make test runs in under a minute with no checkpoint, no network and no Python. It exercises every kernel, the streaming cache, the safetensors reader, the config reader and the tokenizer, then runs three gates over a 13-layer model built from the same tensor graph as the released one: teacher forcing, greedy decode, and incremental decode with the KV cache and carried KDA state. The comparison target is a PyTorch reference captured as fixtures committed to the repo.
That ordering matters. You get a pass or fail on the arithmetic before you spend hours pulling 1.56 TB. A shell script, k3-doctor.sh, does the matching check on the machine itself, sizing your RAM to a preset and measuring your disk the way the engine will read it, and exits non-zero when the hardware cannot run the model at all.
Read the clock honestly
This is a base checkpoint with no chat template, so after "The capital of France is" it continues into quoted text instead of replying. The captured runs are slow by design of the tradeoff: 8 tokens in 261.5 seconds at 32.69 seconds per token on the laptop preset with 8.24 GB peak RSS, and 28 tokens in 299.3 seconds at 10.69 seconds per token on the server preset with 127.92 GB peak. The project also reports that version 1.0.0 made the math per token about 8 times lighter on that hardware, a follow-up chat turn 3.9 times faster, and long prompts roughly half as costly. Those are its own measurements on its own machine, not a general claim about the technique.
None of this is a serving path. What it is is a demonstration that the resident working set can be a parameter you choose rather than a constant the architecture imposes, with output equality held fixed across the whole range.
GitHub: https://github.com/FareedKhan-dev/kimi-k3-in-c
Curated by Agent Palisade — practical AI for small and mid-sized businesses.
