Zero-Trust Linux: How I Hardened 4 Cloud Servers Against Brute-Force Attacks in Under an Hour

No passwords, no root login, and aggressive Fail2ban jails: transforming noisy, attacked ports into impenetrable fortresses.

Spin up a fresh Linux virtual machine on any major cloud provider, assign it a public IPv4 address, and wait ten minutes.

You do not need to advertise the IP address. You do not need to register a domain name. Within moments, automated scanners, compromised routers, and botnets will locate port 22 and begin hammering your operating system with dictionary attacks.

I recently audited four cloud VPS instances deployed across different hosting providers and data centers. These servers ran various workloads, from reverse proxies to microservice APIs. When I opened the authentication logs on the first server, the volume of automated aggression was staggering.

In a twenty-four-hour window, a single machine recorded over fourteen thousand failed authentication attempts. Scripts from hundreds of distinct IP subnets were cycling through usernames like root, admin, ubuntu, test, postgres, and deploy. Every few seconds, the OpenSSH daemon was forced to spawn a new process, negotiate a cryptographic handshake, check authentication credentials, and log a failure.

While modern public-key cryptography protects you against brute-force guessing if passwords are disabled, leaving the default OpenSSH configuration untouched is reckless. Unthrottled connection attempts consume CPU cycles, exhaust connection state tables, clutter system journals, and leave the door open if an operator accidentally creates a user account with a weak password.

In less than sixty minutes, I applied a standardized, zero-trust hardening strategy across all four machines. We eliminated password authentication entirely, barred the root account from remote login, tuned OpenSSH connection boundaries, and deployed persistent Fail2ban jails to ban repeat offenders at the firewall layer.

Here is the exact technical blueprint, step by step, with zero fluff.

The Log Evidence: Quantifying the Attack Surface

Before modifying a single configuration file, inspect the scale of unauthorized traffic hitting your server. You can extract this data directly from the systemd journal or the traditional authentication log.

On Debian and Ubuntu systems running modern systemd, query the SSH service journal for failed password attempts recorded since midnight:

journalctl -u ssh -S today | grep "Failed password" | awk '{print $(NF-3)}' | sort | uniq -c | sort -nr | head -n 10

On CentOS, RHEL, or Rocky Linux systems where the unit is named sshd, query the service like this:

journalctl -u sshd -S today | grep "Failed password" | awk '{print $(NF-3)}' | sort | uniq -c | sort -nr | head -n 10

Sample output from one of our audited nodes:

   3412 185.220.101.5
   2189 194.26.29.112
   1874 45.148.10.74
   1205 103.152.220.44
    982 91.240.118.234
    841 167.99.142.12

Notice what is happening. A single external IP address attempted more than thirty-four hundred logins in less than twelve hours.

Next, check the usernames these bots are attempting:

journalctl -u ssh -S today | grep "Invalid user" | awk '{print $NF}' | sort | uniq -c | sort -nr | head -n 10

Output:

   4520 admin
   3110 root
   1890 test
   1420 user
   1105 ubuntu
    840 oracle
    610 ftpuser
    490 git

Every single one of these attempts represents an unauthenticated entity executing code against your server. If any local account has an empty password, a default cloud password, or a common dictionary phrase, your machine is compromised.

Why Direct Edits to sshd_config Are an Anti-Pattern

For years, the standard tutorial advice was simple: open /etc/ssh/sshd_config with an editor, change three lines, save the file, and restart the daemon.

In modern production environments, editing /etc/ssh/sshd_config directly is a bad practice.

Whenever your package manager updates the OpenSSH server package during regular security upgrades (apt upgrade or dnf update), the package manager detects that the default vendor configuration file was modified. It stops the automated upgrade or throws an interactive prompt asking whether you want to keep your modified file, accept the package maintainer version, or inspect a diff. In automated CI/CD pipelines, Ansible runs, or unattended upgrades, this conflict can stall updates or accidentally revert your security baseline.

OpenSSH version 8.2 and newer solved this problem by introducing modular configuration directories. Look at the very top of /etc/ssh/sshd_config:

# This is the sshd server system-wide configuration file.
Include /etc/ssh/sshd_config.d/*.conf

The daemon reads files inside /etc/ssh/sshd_config.d/ in alphabetical order before parsing the rest of /etc/ssh/sshd_config. Directives set in earlier drop-in files take precedence.

By isolating your security hardening rules inside a dedicated drop-in file like /etc/ssh/sshd_config.d/99-hardening.conf, you keep vendor defaults pristine. Your custom configuration survives package upgrades cleanly and can be deployed across hundreds of machines using a single file transfer.

The Four Core Hardening Directives

To secure our four cloud servers, we created a single drop-in file that implements four critical OpenSSH directives alongside strict protocol settings.

Here is the exact configuration we deployed to /etc/ssh/sshd_config.d/99-hardening.conf:

# /etc/ssh/sshd_config.d/99-hardening.conf
# Production OpenSSH Hardening Baseline

# 1. Disable remote root login completely
PermitRootLogin no

# 2. Enforce cryptographic public key authentication
PubkeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication no

# 3. Limit authentication attempts per connection
MaxAuthTries 3

# 4. Limit unauthenticated connection concurrency
MaxStartups 10:30:60

# 5. Disable unused, high-risk protocol features
X11Forwarding no
AllowTcpForwarding no
AllowAgentForwarding no

# 6. Dead connection detection and idle session teardown
ClientAliveInterval 300
ClientAliveCountMax 2

Let us dissect each of these directives and understand how they alter daemon behavior under the hood.

1. PermitRootLogin no

In default cloud installations, providers often set PermitRootLogin prohibit-password. While this allows public-key access only and stops password guessing for the root account, it still allows external attackers to test whether an SSH key can authenticate directly as root.

Setting PermitRootLogin no enforces an absolute barrier. No user can log into the server as root over the network, regardless of the credentials they present.

To administer the machine, an engineer must first authenticate using a dedicated, unprivileged personal account with an SSH key, and run privileged commands with sudo. This creates a reliable audit trail in /var/log/auth.log that ties every privileged action to an individual identity rather than an anonymous root session.

2. PasswordAuthentication no & KbdInteractiveAuthentication no

Disabling password authentication shuts down dictionary attacks at the cryptographic negotiation phase.

When a client initiates an SSH connection, the server sends an SSH_MSG_USERAUTH_REQUEST listing the authentication methods it supports. With PasswordAuthentication no and KbdInteractiveAuthentication no, the server refuses to offer password or keyboard-interactive mechanisms.

If an attacker does not hold a private key corresponding to an authorized public key stored in ~/.ssh/authorized_keys, the server terminates the connection without ever prompting for an input string.

Use modern Ed25519 keys rather than legacy RSA keys. Generate them on your local workstation with:

ssh-keygen -t ed25519 -C "admin@company.internal"

Ed25519 keys are faster, provide stronger cryptographic security, and avoid the mathematical vulnerabilities that have plagued older RSA key lengths.

3. MaxAuthTries 3

By default, OpenSSH allows up to six failed authentication attempts per connection before terminating the TCP socket.

Why lower this to 3?

Many developers and operations engineers maintain dozens of SSH keys inside their local SSH agent (ssh-agent). When an engineer connects to a server, their SSH client automatically attempts to authenticate using every key loaded in memory, one after another. If the client offers six irrelevant keys before reaching the correct key, the server drops the connection with Too many authentication failures.

Setting MaxAuthTries 3 limits how many keys or credentials an incoming connection can test. This forces clients to use explicit key definitions (configured via ~/.ssh/config using the IdentitiesOnly yes directive) while instantly dropping automated scripts that attempt to cycle through multiple stolen credentials in a single TCP session.

4. MaxStartups 10:30:60

When an automated botnet targets port 22, it does not connect one session at a time. It opens hundreds of concurrent, unauthenticated TCP connections simultaneously to consume server memory and file descriptors.

The MaxStartups directive mitigates this behavior. The syntax 10:30:60 defines three operational thresholds:

• 10: The server allows up to ten unauthenticated connections without restriction.
• 30: Once unauthenticated connections exceed ten, the server begins dropping new connection attempts randomly with a thirty percent probability.
• 60: If unauthenticated connections reach sixty, the server drops all subsequent unauthenticated connection attempts immediately.

This simple parameter prevents connection exhaustion attacks from denying legitimate SSH access to engineers during an ongoing brute-force wave.

The Pre-Flight Safety Verification

Before applying these changes to a live remote server, follow the golden rule of remote system administration:

Never reload or restart the SSH service without testing the syntax first, and never close your active terminal session until you have verified a successful secondary login.

If you introduce a typo into an SSH configuration file and restart the daemon, the service will fail to bind to port 22. If your current session disconnects, you are locked out of the machine and must resort to your cloud provider's web console to fix the mistake.

Run the OpenSSH syntax validator:

sudo sshd -t

The -t flag runs OpenSSH in test mode. It reads all included files, validates configuration syntax, checks permission boundaries, and returns cleanly with exit code 0 if everything is valid. If there is a syntax error, it prints the exact file name and line number where the issue occurred.

Once sshd -t passes with zero errors, reload the daemon:

sudo systemctl reload ssh

Note that we use reload, not restart.

Reloading sends a SIGHUP signal to the parent OpenSSH process. The parent re-reads the configuration files and applies them to future incoming connections, without interrupting or terminating existing, active SSH sessions.

Now, keep your current terminal window open. Open a completely new terminal window on your local machine and test authenticating into the server:

ssh -i ~/.ssh/id_ed25519 deploy@server_ip

If the secondary terminal connects successfully without prompting for a password, your configuration is rock-solid.

Deploying Fail2ban for Persistent Firewall Jails

Eliminating password authentication secures your server from unauthorized entry, but it does not stop the noise.

Even with public key authentication enforced, unauthorized connection attempts still hit port 22, trigger TCP three-way handshakes, spawn child processes, and generate thousands of log lines.

To eliminate this noise, deploy Fail2ban. Fail2ban monitors log files for repeated authentication failures and dynamically adds firewall rules using nftables or iptables to drop traffic from offending IP addresses before the packets ever reach OpenSSH.

Install Fail2ban:

On Debian and Ubuntu:

sudo apt update && sudo apt install -y fail2ban

On RHEL, Rocky Linux, or AlmaLinux:

sudo dnf install -y epel-release && sudo dnf install -y fail2ban

Configuring Fail2ban Jails the Right Way

Fail2ban ships with a default configuration file located at /etc/fail2ban/jail.conf.

Just like OpenSSH, you should never edit jail.conf directly. Future package updates will overwrite your changes. Instead, create a local override file named /etc/fail2ban/jail.d/01-sshd.local or /etc/fail2ban/jail.local.

Create /etc/fail2ban/jail.d/01-sshd.local:

[DEFAULT]
# Whitelist local loopback and your trusted office IP
ignoreip = 127.0.0.1/8 ::1 203.0.113.50

# Default ban duration (1 hour)
bantime = 1h

# Window of time to measure failed attempts (10 minutes)
findtime = 10m

# Maximum failed attempts within findtime before a ban is issued
maxretry = 3

# Enable progressive, exponential bantime for repeat offenders
bantime.increment = true
bantime.factor = 2
bantime.maxtime = 48h

[sshd]
enabled = true
port = ssh
filter = sshd
backend = systemd
maxretry = 3

Notice the key configuration choices we made:

• ignoreip: Crucial. Always add your static office IP, home VPN, or bastion host to this line. This prevents you or your team from accidentally banning yourselves if someone runs an automated deployment script with misconfigured keys.
• backend = systemd: On modern Linux distributions where logging is handled by systemd-journald, using the systemd backend is significantly faster and more reliable than tailing disk-based flat files in /var/log/auth.log.
• bantime.increment = true: This enables exponential backoff. If an IP gets banned once, it is blocked for one hour. If it returns and triggers another ban, the ban duration doubles to two hours, then four, up to forty-eight hours. This silences persistent botnets indefinitely.

Start and enable the Fail2ban service:

sudo systemctl enable --now fail2ban

Verifying Jail Status and Real-Time Bans

Once Fail2ban is running, verify that the SSH jail is actively monitoring the daemon:

sudo fail2ban-client status sshd

Sample output after fifteen minutes of operation on one of our cloud nodes:

Status for the jail: sshd
|- Filter
|  |- Currently failed: 4
|  |- Total failed:     284
|  `- File list:        systemd
`- Actions
   |- Currently banned: 18
   |- Total banned:     37
   `- Banned IP list:   45.148.10.74 91.240.118.234 185.220.101.5 194.26.29.112 ...

Notice the immediate impact. Eighteen abusive IP addresses were banned at the packet filter level.

You can inspect the exact kernel firewall rules created by Fail2ban. On systems using nftables:

sudo nft list ruleset | grep -A 10 "f2b-sshd"

Or on systems running legacy iptables:

sudo iptables -S | grep f2b-sshd

When packets arrive from any banned IP address, the Linux kernel drops them immediately at the network interface layer. The packets never trigger socket allocations, OpenSSH never spawns a process, and your authentication log remains completely quiet.

How to Unban an IP Address

If an engineer accidentally triggers a ban, do not reboot the server or restart the firewall. Unban the specific IP address cleanly using the fail2ban-client:

sudo fail2ban-client set sshd unbanip 198.51.100.25

The tool deletes the corresponding firewall rule instantly and restores network connectivity for that IP.

The 60-Second Hardening Script

To repeat this configuration across multiple machines without typing individual commands, you can assemble the steps into a simple, auditable bash script.

Here is the script we used to roll out this configuration across the remaining three nodes:

#!/usr/bin/env bash
set -euo pipefail

echo "==> Applying OpenSSH Hardening Baseline..."

cat <<'EOF' | sudo tee /etc/ssh/sshd_config.d/99-hardening.conf > /dev/null
# Production OpenSSH Hardening Baseline
PermitRootLogin no
PubkeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication no
MaxAuthTries 3
MaxStartups 10:30:60
X11Forwarding no
AllowTcpForwarding no
ClientAliveInterval 300
ClientAliveCountMax 2
EOF

# Validate syntax
sudo sshd -t
echo "OpenSSH syntax valid."

# Reload OpenSSH cleanly
sudo systemctl reload ssh || sudo systemctl reload sshd
echo "OpenSSH reloaded."

echo "==> Configuring Fail2ban SSH Jail..."
cat <<'EOF' | sudo tee /etc/fail2ban/jail.d/01-sshd.local > /dev/null
[DEFAULT]
ignoreip = 127.0.0.1/8 ::1
bantime = 1h
findtime = 10m
maxretry = 3
bantime.increment = true

[sshd]
enabled = true
port = ssh
filter = sshd
backend = systemd
maxretry = 3
EOF

sudo systemctl restart fail2ban
echo "Fail2ban deployed and active."

An Interesting Fact About SSH Port 22

Why does SSH run on port 22 by default?

In July 1995, Tatu Ylönen, a researcher at Helsinki University of Technology, designed the original Secure Shell protocol after a password-sniffing attack compromised his university network.

At the time, the dominant remote login tools were FTP (running on port 21) and Telnet (running on port 23). Ylönen wanted SSH to replace both insecure protocols. He noticed that port 22 between them was unassigned.

He sent an email to Jon Postel, the legendary Internet Assigned Numbers Authority (IANA) administrator who managed global port assignments. Postel approved the request and assigned port 22 to SSH in a single day.

Within weeks, the protocol spread across the nascent internet, and port 22 has been the standard target for network scanners ever since.

Summary and Operating Checklist

Hardening Linux infrastructure does not require complex, invasive tooling or heavy enterprise software agents.

By applying foundational zero-trust principles to OpenSSH, you eliminate the entire vector of credential guessing and brute-force intrusion:

• Never edit /etc/ssh/sshd_config directly; use /etc/ssh/sshd_config.d/*.conf drop-ins to survive package upgrades cleanly.
• Bar root logins with PermitRootLogin no and mandate unprivileged accounts with sudo.
• Enforce cryptographic keys with PasswordAuthentication no and modern Ed25519 pairs.
• Limit authentication attempts with MaxAuthTries 3 and throttle scanners with MaxStartups 10:30:60.
• Test before applying with sshd -t and prefer systemctl reload ssh over destructive restarts.
• Drop repeat offenders at the firewall layer using Fail2ban with incremental backoff.

After applying these settings across our four cloud instances, failed authentication attempts dropped from over fourteen thousand per day to zero. The system journals are readable, CPU usage is flat, and the perimeter is secure.

How many failed authentication attempts is your primary production server logging right now?

If this breakdown saved you hours of debugging or gave you something practical to use in production, consider buying me a coffee. Your support directly fuels independent, zero-fluff Linux and DevOps technical guides.

About the Author

Asep Sayyad is a Linux and DevOps engineer passionate about Linux administration, automation, cloud technologies, containers, and open-source software. He enjoys solving real-world infrastructure challenges and sharing practical knowledge through in-depth technical articles, tutorials, and hands-on guides.

His goal is to help aspiring and experienced engineers build stronger Linux and DevOps skills with content focused on real production scenarios rather than theory alone.

Connect with Me

• Portfolio: asepsayyad007.in
• Blog: asepsayyad007.in/blogs
• GitHub: github.com/asepsayyad007
• LinkedIn: linkedin.com/in/asepsayyad
• Medium: asepsayyad007.medium.com
• Support: buymeacoffee.com/asepsayyad007

Enjoyed this article?

If this guide saved you hours of debugging or gave you something practical for production, consider:

• Buying me a coffee: Your support directly fuels independent, zero-fluff Linux and DevOps engineering breakdowns.
• Starring my open-source projects on GitHub.
• Sharing this article with fellow Linux and DevOps engineers.

You can also follow me for more practical content on Linux, DevOps, Cloud, Containers, Automation, and Open Source. Thanks for reading, and enjoy your learning!

© 2026 Asep Sayyad