54 Containers, 93 Services, 64 GB of RAM: What Actually Eats It

The Local LLM Lab — Part 11

One controlled experiment. One number. One verdict.

A home server has a question the cloud never asks: who is eating the RAM?

In the cloud, memory is a line item on a bill. At home, it's a finite pool — 64 GB on my box — shared by every container, every service, every daemon, and the one GPU that runs the AI. When the pool runs dry, something gets killed, and the thing that gets killed is usually the thing you least expected.

I ran a memory audit on my homelab: 54 Docker containers, 93 running systemd units, one RTX 3090, 64 GB of RAM. I pulled the per-service memory history over 7 days, the current per-process state, and the kernel's non-reclaimable share. This is what the 64 GB is actually holding, and what the number says about how much RAM a homelab really needs.

The setup

• Machine: HP Z840 Workstation, 2× Intel Xeon E5-2680 v4 (28 cores, 56 threads), 64 GB RAM, one RTX 3090
• Workload: 54 Docker containers, 93 running systemd units, all up
• Telemetry: 7 days of per-service memory at 2-hour resolution, plus the current per-process state and the kernel's non-reclaimable memory
• GPU tenants (for context): Whisper ASR, a TTS model, an embedding model — see the VRAM piece for what the card holds

The 64 GB is the honest number. It's not a "big" machine by data-centre standards, but it's a real workstation, and it runs a real stack: photo storage, music, a RAG pipeline, an LLM observability stack, a CI runner, a media server, a few self-hosted apps, and the AI services on the GPU.

The number

The box is using 41.6 GB of its 64 GB — 65% — and the peak over the week was 15.9 GB of application memory, with the rest held by the kernel and the GPU's system-side buffers.

The "other 49 containers + 93 services" line is the one that surprises people. It's 35% of the RAM, and it's not one thing — it's a long tail. The biggest individual entries in that tail:

The pattern: no single service is the hog. The RAM is eaten by a long tail of small, resident services, each holding 10–1,000 MB, and the sum is 22.7 GB. The GPU tenants hold the most per service (7.7 GB and 5.0 GB), but they're two services. The long tail is forty-nine, and it's still a third of the box.

What the peak says

The 7-day peak of 15.9 GB of application memory is the number that matters for sizing. It happened when the photo server was running a batch of thumbnail generation at the same time the CI runner was building a project and the observability stack was ingesting a burst of traces. Three things spiking at once, on a 64 GB box, and the peak was 15.9 GB of application memory — 25% of the pool.

The headroom at peak was 48 GB. That's the insurance. The box has never come close to the ceiling, and the events log over the week shows zero OOM kills, zero memory threshold crossings — the only alerts were eight disk I/O spikes, which are storage, not memory.

The honest reading: 64 GB is comfortable for this stack. 32 GB would be tight but workable. 16 GB would not. The peak of 15.9 GB is the worst case that actually happened, and it left 48 GB on the table. If I were buying RAM for this box today, I'd buy 64 GB and stop — not because it's the minimum, but because it's the size where the peak is a non-event.

The counterargument: RAM is cheap, don't sweat it

The fair objection: RAM is the cheapest part of a server. 64 GB is a few hundred euros. Why spend an article on it?

Because the RAM question is where homelab sizing goes wrong. People size the GPU (the expensive, scarce part) and under-size the RAM (the cheap, abundant part), and then the box OOMs the one service that matters — usually the database or the photo server — at the worst moment. The GPU is the part everyone benchmarks. The RAM is the part nobody measures, and it's the part that fails.

The measurement says the failure mode is not "one big service." It's the long tail. Fifty small services, each holding 10–100 MB, add up to a third of the box. The fix is not "buy more RAM" (though that helps). The fix is knowing the tail is real, so when the box gets tight you know which of the fifty to trim, not which of the two to kill.

What I would do now

If I were building this box today:

• RAM: 64 GB. The measured peak is 15.9 GB of application memory; 64 GB is 4× peak. 32 GB would be 2× peak — workable, but the photo server and the CI runner would start fighting.
• The rule: size RAM for the peak of the long tail, not for the biggest single service. The biggest service is 7.7 GB. The tail is 22.7 GB. The tail is the size.
• The audit: a per-service memory panel, current and 7-day peak, with the kernel share separated out. The number that matters is not "the box is at 65%" — it's "the peak was 15.9 GB, and here are the ten services that were up at the peak."

The homelab monitor I use has exactly that panel. The 7-day peak per service, the current per-process state, and the kernel's non-reclaimable share, in one view. It's the difference between "the box is full" and "here's what's full, and here's what to trim."

Verdict

54 containers, 93 services, 64 GB of RAM: 41.6 GB in use (65%), 15.9 GB peak over the week, zero OOM kills. The RAM is not eaten by one big service — it's eaten by a long tail of fifty small, resident services that add up to 22.7 GB. The GPU tenants hold the most per service, but the tail holds a third of the box by itself. Size the RAM for the peak of the tail, not for the biggest single service, and the box stops being a guessing game.

The Local LLM Lab is a series of measured experiments on a home GPU stack. Every number in this article was pulled from 7 days of per-service memory telemetry on the box, not estimated. Next: the one failed service on a 93-service stack — and how the uptime checks keep the rest alive.