It was supposed to be a relaxed evening. I had finally convinced my girlfriend to watch the Lord of the Rings Extended Edition with me. The mood was good, the elves were marching into Helm's Deep. But right around 10 PM, everything went dark.
The entire LAN was dead. No streaming on the TV and no internet on the PC. The strange thing was that the Wi-Fi on my Fritzbox 7690 was still working perfectly. Only everything connected via cable was completely knocked out.
The Suspect in the Server Closet
Since I had just recently put the Lenovo M920q into service, it was naturally my prime suspect. My first assumption was a misconfigured Docker container. I thought about problems with macvlan or a container in Network Host Mode that might be causing a loop.
I disabled a suspicious container and had two days of peace. I thought the problem was solved. But then it happened again: right in the middle of the movie, the LAN was gone. Only when I pulled the network cable from the Lenovo did the entire network calm down immediately.
Hunting Through the Kernel Logs
I SSHed into the Proxmox host and searched through the logs:
dmesg -T | grep -i e1000e
There I found error messages appearing every second:
e1000e 0000:00:1f.6 nic0: Detected Hardware Unit Hang
TDH <52>
TDT <73>
next_to_use <73>
next_to_clean <52>
Technically, this is a problem with the ring buffers of the network card. Think of it like a conveyor belt. The system puts packets on the back (TDT - Transmit Descriptor Tail) and the card sends them off the front (TDH - Transmit Descriptor Head).
In my case, the logs showed:
- TDH (Head): 52
- TDT (Tail): 73
This means the system is queueing packets, but the hardware isn't processing them on the front end. The network card is simply frozen internally. Because the server is connected to the LAN via a bridge, this hang clogs up the entire Fritzbox switch port and takes down the rest of the wired network with it.
Why Does This Happen?
The culprit is a feature called EEE (Energy Efficient Ethernet), also known as IEEE 802.3az. The Intel i219-LM card tries to save power when there's little activity. But in combination with virtualization under Proxmox, it sometimes doesn't wake up properly. This leads to timing problems and ends in a Hardware Unit Hang.
The Solution
I applied three steps to get the Lenovo stable again:
1. Adjust Driver Options
In /etc/modprobe.d/e1000e.conf I added the following line:
options e1000e InterruptThrottleRate=0,0,0 EEE=0
This disables Energy Efficient Ethernet and interrupt throttling at the driver level.
2. Disable Offloading
So-called offloading refers to functions where the card takes work off the CPU (TSO, GSO, GRO). That sounds good in theory, but often leads to problems:
ethtool -K nic0 tso off gso off gro off
3. Create a Systemd Service
To make these settings persist after every reboot, I created a systemd service:
/etc/systemd/system/nic-stable.service:
[Unit]
Description=Disable NIC offloading and EEE for stability
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/sbin/ethtool -K nic0 tso off gso off gro off
ExecStart=/usr/sbin/ethtool -s nic0 eee off
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target
Then activated it:
systemctl daemon-reload
systemctl enable nic-stable.service
systemctl start nic-stable.service
And an initramfs update to load the modprobe config:
update-initramfs -u
The Rescue Mission Conclusion
It wasn't a bug in Proxmox or the Fritzbox. It was simply the hardware losing the thread while trying to save power. By the way, the Fritzbox doesn't see any of this because the problem happens at Layer 2 the network card just hangs before any IP packets can even be sent.
Since these adjustments, the system has been running without a single dropout. We were finally able to finish Lord of the Rings without interruption. If your LAN ever goes down for no apparent reason, take a look at your server's kernel logs:
dmesg -T | tail -100
Often it's just a network card that slept a little too deeply. 🛌💤
