Stop Thrashing Under Memory Pressure: Practical zram + systemd-oomd on Linux

When a homelab box or small VPS runs out of free RAM, the failure mode is rarely a clean kill. More often the machine spends minutes thrashing: anonymous pages bounce to a slow disk swap, the page cache collapses, SSH becomes sticky, and then the kernel OOM killer finally fires.

Two complementary tools fix different halves of that story:

1. zram — a compressed block device in RAM, usually used as high-priority swap so reclaim stays in memory instead of on disk.
2. systemd-oomd — a userspace OOM daemon that watches cgroup v2 pressure stall information (PSI) and kills a descendant cgroup before the whole host livelocks.

This post is a practical setup for both, with verification and a clean rollback. It is not a hibernation guide, not a zswap deep dive, and not another MemoryMax= sandbox recipe.

What you are actually fixing

Swap is not “emergency RAM.” Chris Down’s well-known write-up makes the real point: swap exists so rarely used anonymous pages can be reclaimed the same way clean file pages can. Without any swap, those anonymous pages stay pinned, reclaim is less egalitarian, and under pressure you thrash the page cache instead of a swap device.

Disk-backed swap still works — but on a busy SSD or a slow VPS volume it can turn moderate pressure into multi-second stalls. zram keeps that swap path in RAM with compression (kernel docs note a rough ~2:1 expectation; real ratios vary by workload). systemd-oomd then uses the breathing room that swap creates so it can react on PSI instead of waiting for the global kernel OOM path.

Memory pressure
      │
      ├─► reclaim cold anon pages → zram swap (compressed RAM)
      │
      └─► sustained PSI / swap exhaustion → systemd-oomd SIGKILL of a leaf cgroup

Prerequisites

• Linux with the zram module (common on modern distros).
• cgroup v2 unified hierarchy (systemd default on current Debian/Ubuntu/Fedora/Arch).
• Kernel PSI support (mainline since 4.20): /proc/pressure/memory must exist.
• Packages:
  • zram-generator (Fedora ships it by default on many spins; Arch package zram-generator; Debian/Ubuntu package name is typically zram-tools or install upstream/distro systemd-zram-generator / zram-generator depending on release — confirm with your package manager).
  • systemd-oomd (package systemd-oomd on Debian/Ubuntu; often already present on Fedora).

Check the basics:

# cgroup v2?
mount | grep -E 'cgroup2|type cgroup2'

# PSI present?
cat /proc/pressure/memory

# zram module available?
modinfo zram | head

Part 1 — Compressed swap with zram-generator

Why the generator instead of a one-shot script

zram-generator is a systemd unit generator. You drop a small conf file; at boot it creates systemd-zram-setup@zramN.service, formats the device (swap by default), and activates it. No fragile rc scripts, no hand-rolled mkswap in rc.local.

Config path precedence (lowest to highest override style matching systemd norms):

• /usr/lib/systemd/zram-generator.conf
• /etc/systemd/zram-generator.conf ← administrator file
• drop-ins under *.conf.d/

Kernel cmdline systemd.zram=0 disables generator devices; systemd.zram=1 forces zram0 with defaults.

Minimal working config

sudo tee /etc/systemd/zram-generator.conf >/dev/null <<'EOF'
[zram0]
# Uncompressed capacity as a function of MemTotal (MiB variable: ram).
# Default if omitted: min(ram / 2, 4096)
zram-size = min(ram / 2, 8192)

# Prefer a fast modern compressor when the kernel offers it.
compression-algorithm = zstd

# Higher than typical disk swap so zram is chosen first.
swap-priority = 100

# Default options already include discard; keep it explicit.
options = discard
EOF

Notes from the man page / upstream docs:

After writing the conf:

sudo systemctl daemon-reload
sudo systemctl start systemd-zram-setup@zram0.service
systemctl status systemd-zram-setup@zram0.service --no-pager

Verify zram swap

zramctl
swapon --show
cat /proc/swaps

# Live compression stats (fields documented in the kernel zram guide)
cat /sys/block/zram0/mm_stat
cat /sys/block/zram0/comp_algorithm

zramctl columns to care about:

• DISKSIZE — uncompressed capacity you configured.
• DATA — uncompressed bytes currently stored.
• COMPR — compressed payload size.
• TOTAL — RAM actually used including allocator overhead.

Disable zswap when zram is the primary swap

If the kernel’s zswap pool is enabled, it sits in front of swap devices as a compressed cache. On many stock kernels it is on by default. Arch’s zram page is explicit: leaving zswap enabled can intercept pages before they reach zram and waste the setup.

Runtime check and disable:

# Current state (Y/N or 1/0 depending on kernel)
cat /sys/module/zswap/parameters/enabled 2>/dev/null || echo 'zswap module params not present'

# Temporary disable
echo 0 | sudo tee /sys/module/zswap/parameters/enabled

Persist with a kernel parameter (bootloader-specific):

zswap.enabled=0

Or keep zswap and do not use zram — pick one primary strategy. Mixing both without measuring usually confuses operators more than it helps.

VM sysctls that make sense for in-memory swap

Disk swap wants conservative swappiness. Compressed RAM swap is different: reclaiming into zram is often cheaper than dropping hot file cache. The kernel’s own vm.swappiness documentation notes that for in-memory swap, values beyond 100 can be reasonable.

A widely copied starting point (Pop!_OS defaults / community zram benchmarks summarized on the ArchWiki):

sudo tee /etc/sysctl.d/99-vm-zram-parameters.conf >/dev/null <<'EOF'
# Bias reclaim toward anon pages when swap is fast (zram).
vm.swappiness = 180

# Disable watermark boosting (can cause sudden reclaim spikes).
vm.watermark_boost_factor = 0

# Wake kswapd earlier; scale is in fractions of 10000.
vm.watermark_scale_factor = 125

# Disable swap readahead (page-cluster is log2 pages). zram has no disk locality.
vm.page-cluster = 0
EOF

sudo sysctl --system

Treat these as a starting point, not religion. Measure with your workload.

Optional: resident cap so zram cannot eat the host

If you want a hard ceiling on compressed memory used by zram:

[zram0]
zram-size = ram / 2
zram-resident-limit = ram / 4
compression-algorithm = zstd
swap-priority = 100

That maps to the kernel mem_limit sysfs knob described in the zram admin guide.

Hibernation warning

Hibernation to a zram swap device is not supported. logind will refuse it. If you hibernate, you still need a disk-backed swap large enough for the image — separate from this guide.

Part 2 — Early kills with systemd-oomd

What oomd actually does

From systemd-oomd.service(8):

• Runs in userspace on cgroup v2 + PSI.
• Units opt in with ManagedOOMSwap=kill and/or ManagedOOMMemoryPressure=kill.
• When limits trip, oomd sends SIGKILL to processes in a descendant cgroup (not usually the monitored unit itself).
• Only leaf cgroups, or cgroups with memory.oom.group=1, are candidates.
• Swap is strongly recommended. Without swap, pressure spikes are abrupt and oomd has less time to act; swap-based actions are ignored if there is no swap.

Global knobs live in oomd.conf(5):

Install and enable

Debian/Ubuntu:

sudo apt install systemd-oomd
sudo systemctl enable --now systemd-oomd.service
systemctl status systemd-oomd.service --no-pager

Fedora often already enables it; confirm with the same systemctl commands.

Memory accounting should be on (modern systemd defaults are usually fine):

# system-wide default
systemctl show -p DefaultMemoryAccounting

If it is false on an older image:

sudo mkdir -p /etc/systemd/system.conf.d
sudo tee /etc/systemd/system.conf.d/10-memory-accounting.conf >/dev/null <<'EOF'
[Manager]
DefaultMemoryAccounting=yes
EOF
sudo systemctl daemon-reexec

Recommended slice policy

systemd’s own usage notes:

• Put ManagedOOMSwap=kill on a high ancestor (often -.slice via a drop-in) so swap exhaustion can pick the worst descendant.
• Put ManagedOOMMemoryPressure=kill on slices below the root — e.g. user.slice and/or system.slice — with tighter limits for interactive user sessions.

# Swap watchdog at the root
sudo mkdir -p /etc/systemd/system/-.slice.d
sudo tee /etc/systemd/system/-.slice.d/20-oomd-swap.conf >/dev/null <<'EOF'
[Slice]
ManagedOOMSwap=kill
EOF

# Pressure watchdog for user sessions (stricter limit)
sudo mkdir -p /etc/systemd/system/user.slice.d
sudo tee /etc/systemd/system/user.slice.d/20-oomd-pressure.conf >/dev/null <<'EOF'
[Slice]
ManagedOOMMemoryPressure=kill
ManagedOOMMemoryPressureLimit=40%
EOF

# Pressure watchdog for system services (more tolerant)
sudo mkdir -p /etc/systemd/system/system.slice.d
sudo tee /etc/systemd/system/system.slice.d/20-oomd-pressure.conf >/dev/null <<'EOF'
[Slice]
ManagedOOMMemoryPressure=kill
ManagedOOMMemoryPressureLimit=60%
EOF

sudo systemctl daemon-reload

Optional global tuning drop-in:

sudo mkdir -p /etc/systemd/oomd.conf.d
sudo tee /etc/systemd/oomd.conf.d/20-local.conf >/dev/null <<'EOF'
[OOM]
# Act a bit earlier on small boxes; raise if you see flapping kills.
SwapUsedLimit=80%
DefaultMemoryPressureLimit=50%
DefaultMemoryPressureDurationSec=20s
EOF

sudo systemctl reload systemd-oomd.service 2>/dev/null || sudo systemctl restart systemd-oomd.service

Protect critical units from being preferred victims:

# drop-in on e.g. ssh.service
[Service]
ManagedOOMPreference=avoid

omit is stronger than avoid (requires xattr support; see systemd.resource-control(5) ownership rules).

Inspect oomd state

oomctl dump
journalctl -u systemd-oomd.service -b --no-pager | tail -n 50

Dry-run mode exists on newer systemd builds (systemd-oomd --dry-run / unit drop-in) if you want log lines without kills while testing — check your installed man page for --dry-run (added in systemd 253).

Part 3 — Read pressure like an operator

PSI exports stall ratios under /proc/pressure/:

cat /proc/pressure/memory
# some avg10=... avg60=... avg300=... total=...
# full avg10=... avg60=... avg300=... total=...

• some — at least some tasks stalled on memory.
• full — non-idle tasks essentially all stalled (thrashing territory).

Per-cgroup files live at memory.pressure inside the cgroupfs. systemd-oomd is basically an automated consumer of those signals with a kill policy attached.

Quick health snapshot:

echo '=== mem ===' ; free -h
echo '=== swaps ===' ; swapon --show
echo '=== zram ===' ; zramctl
echo '=== psi ===' ; cat /proc/pressure/memory
echo '=== oomd ===' ; oomctl dump | head -n 80

Safe soak test (use a throwaway cgroup)

Do not OOM-test a production host. On a lab machine or VM:

# Optional: watch PSI in another terminal
watch -n1 cat /proc/pressure/memory

# Allocate until pressure rises (install stress-ng if needed)
sudo systemd-run --scope -p MemoryMax=512M --unit=memtest-scope \
  stress-ng --vm 2 --vm-bytes 95% --timeout 60s

Watch whether:

1. zramctl DATA/COMPR climb instead of disk swap thrashing.
2. oomctl dump shows monitored cgroups and, if limits trip, journal lines from systemd-oomd about kills under memtest-scope / user slices — not a random freeze of the whole machine.

Rollback

# oomd policy
sudo rm -f /etc/systemd/system/-.slice.d/20-oomd-swap.conf
sudo rm -f /etc/systemd/system/user.slice.d/20-oomd-pressure.conf
sudo rm -f /etc/systemd/system/system.slice.d/20-oomd-pressure.conf
sudo rm -f /etc/systemd/oomd.conf.d/20-local.conf
sudo systemctl daemon-reload
sudo systemctl disable --now systemd-oomd.service   # only if you want it fully off

# zram
sudo systemctl stop systemd-zram-setup@zram0.service
sudo rm -f /etc/systemd/zram-generator.conf
sudo systemctl daemon-reload
# optionally remove /etc/sysctl.d/99-vm-zram-parameters.conf and re-run sysctl --system

# re-enable zswap if you disabled it and still want it
echo 1 | sudo tee /sys/module/zswap/parameters/enabled
# and remove zswap.enabled=0 from kernel cmdline if added

Design boundaries (so this stays the right tool)

Practical defaults I actually use

On a 8–32 GiB homelab host:

1. zram-size = min(ram / 2, 8192) with zstd, priority 100.
2. zswap.enabled=0 when zram is primary.
3. vm.swappiness=180, vm.page-cluster=0 as a baseline.
4. systemd-oomd enabled with swap kill on -.slice and pressure kill on user.slice / system.slice.
5. ManagedOOMPreference=avoid on sshd and the backup runner.

On a 1–2 GiB VPS, shrink zram-size with a resident limit (zram-resident-limit = ram / 3) so compression metadata cannot surprise you, and lower DefaultMemoryPressureDurationSec so oomd fails closed faster.

References

• Linux kernel — Compressed RAM-based block devices (zram)
• systemd project — zram-generator.conf(5) / github.com/systemd/zram-generator
• Debian manpages — systemd-oomd.service(8), oomd.conf(5), oomctl(1), systemd.resource-control(5) (ManagedOOM*)
• Kernel — Pressure Stall Information (PSI), Documentation for /proc/sys/vm/ (swappiness, page-cluster, watermarks)
• ArchWiki — zram, zswap
• Chris Down — In defence of swap: common misconceptions
• Pop!_OS default-settings discussion of zram VM tunables (via ArchWiki citations): pop-os/default-settings#163

Takeaway: give the kernel a fast place to put cold anonymous pages (zram), tune reclaim so it actually uses that path, and let systemd-oomd cut the worst cgroup loose when PSI says the box is stalling — before disk thrash turns a recoverable spike into a hard reset.