😩 The part of CI/CD nobody enjoys
You want a GitHub Actions job to rsync a build onto an EC2 box and restart a service. Simple, right?
Then reality shows up:
- 🔑 The key. You generate an SSH key pair, paste the private half into
secrets.SSH_PRIVATE_KEY, and append the public half to~/.ssh/authorized_keyson the instance. That key now lives forever. It never rotates. Anyone who can read repo secrets — or any action youuses:that decides to be clever — has shell on production. - 🌍 Port 22. GitHub-hosted runners come from a huge, changing pool of egress IPs. So you either open
22/tcpto0.0.0.0/0(🙈), or you write a scheduled job that pulls GitHub's meta API and rewrites your security group ingress rules — dozens of CIDRs, churning weekly, on every instance you deploy to. - 🏰 The bastion. The "proper" fix. Now you have an extra instance to patch, monitor, pay for, and whose own key you also have to manage. Congratulations, the problem has a second copy of itself.
- 🧟 The leftovers. A public IP on a box that has no business having one. A key on an ex-employee's laptop. A
known_hostsprompt that hangs a job at 2am.
Every one of these is accepted as "just how deploys work." It isn't, anymore.
⚡ Enter AWS Systems Manager Session Manager
Session Manager flips the direction of the connection. 🔄
The SSM Agent on your instance makes an outbound HTTPS connection to AWS and holds it open. When you want in, you ask the SSM API for a session, and AWS brokers the two ends together over that existing channel.
Read that again, because everything good follows from it:
- 🚫 Zero inbound rules. Security group ingress can be completely empty. Port 22 closed. To everyone. Forever.
- 🕳️ No public IP, no bastion. Private subnet instances work identically. Behind a NAT gateway, or with no internet at all if you add the three VPC interface endpoints.
- 🪪 IAM is the auth layer. Access is an IAM policy, not a file on a disk. Revoke a role and access dies instantly — no hunting for
authorized_keysentries. - 📜 CloudTrail sees every session start, attributed to the identity that opened it.
And the underrated trick: the AWS-StartSSHSession document turns that broker into a raw byte tunnel, which OpenSSH will happily use as a ProxyCommand. Meaning SSH itself still runs — end-to-end encrypted, host keys and all — it just stops caring about routing.
Pair it with EC2 Instance Connect (SendSSHPublicKey) and the last piece falls over too: you push a freshly generated public key into instance metadata, where sshd picks it up for 60 seconds and then forgets it. 🔥 An SSH key with a one-minute shelf life. Nothing to rotate, nothing to leak.
That's the whole idea. The annoying part is the boilerplate: generate a key, push it, write a ProxyCommand block into ~/.ssh/config, get the host-key checking right, and tear it all down afterwards.
So I packaged it. 📦
🚀 The action
ankurk91/setup-ssh-over-ssm-action does the setup and the cleanup. It doesn't wrap ssh — it configures the runner, then gets out of your way.
name: Deploy
on:
push:
branches: [ main ]
permissions:
id-token: write
contents: read
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: aws-actions/configure-aws-credentials@v6
with:
role-to-assume: ${{ vars.AWS_IAM_ROLE_ARN }}
aws-region: us-east-1
- uses: ankurk91/setup-ssh-over-ssm-action@v1
with:
instance-id: ${{ vars.EC2_INSTANCE_ID }}
os-user: ubuntu
- run: ssh ssm-target 'uptime'
No secrets. No key. No security group rule. 🎉 OIDC gets short-lived AWS credentials, the action does the rest.
🧩 What it actually wrote
That middle step generates an ephemeral ed25519 key, pushes it via EC2 Instance Connect, and drops a fenced block at the top of ~/.ssh/config:
Host ssm-target
HostName i-0123456789abcdef0
User ubuntu
Port 22
IdentityFile "/home/runner/.ssh/ssm-ssm-target-i-0123456789abcdef0-9f3c1a2b"
IdentitiesOnly yes
StrictHostKeyChecking accept-new
UserKnownHostsFile "/home/runner/.ssh/ssm-ssm-target-i-0123456789abcdef0-9f3c1a2b.known_hosts"
ServerAliveInterval 30
ControlMaster auto
ControlPath "/home/runner/.ssh/ssm-9f3c1a2b.sock"
ControlPersist 1h
ProxyCommand sh -c "aws ssm start-session --target %h --document-name AWS-StartSSHSession \
--parameters 'portNumber=%p' --region us-east-1 \
--reason 'setup-ssh-over-ssm-action/18273645/1/9f3c1a2b'"
Match all
Two details worth pointing at. The --reason stamp carries the workflow run id, so every tunnel is labelled in the Session Manager console and in CloudTrail — and the post step can find exactly its own sessions later. And Match all closes the stanza, because the block goes in first in the file and must not swallow any global directives you keep above your own Host lines.
Everything else is ordinary SSH config, which is the point: anything that speaks SSH just works, unmodified 🛠️
- run: rsync -az --delete ./build/ ssm-target:/var/www/app/current/
- run: scp ./config/production.env ssm-target:/srv/app/.env
- run: ssh ssm-target 'cd /srv/app && ./bin/migrate --no-interaction'
- run: ssh ssm-target 'sudo systemctl reload nginx'
- run: ansible-playbook -i inventory.yml site.yml # ansible_host: ssm-target
Name the alias whatever reads well in your pipeline:
- uses: ankurk91/setup-ssh-over-ssm-action@v1
with:
instance-id: i-0123456789abcdef0
host-alias: app-server
os-user: ec2-user # Amazon Linux
Bring your own key instead of the ephemeral one — it must have no passphrase, and it's the required route for hybrid mi- managed nodes, which EC2 Instance Connect doesn't support:
- uses: ankurk91/setup-ssh-over-ssm-action@v1
with:
instance-id: mi-0123456789abcdef0
private-key: ${{ secrets.SSH_PRIVATE_KEY }}
🧹 And it cleans up after itself
A post step running on always() removes the config block, deletes the key material, closes the multiplexed master connection, and terminates the SSM sessions carrying this run's marker. Nothing survives the job. 🫧
✅ What you need
On the runner — AWS CLI v2 (recent enough to accept aws ssm start-session --reason) and the Session Manager plugin. Both are preinstalled on GitHub-hosted Ubuntu runners, so: nothing. On self-hosted, add install-aws-cli-action and install-session-manager-plugin-action.
On the instance — Linux, sshd running (the port can stay firewalled shut), SSM Agent ≥ 2.3.672.0, AmazonSSMManagedInstanceCore on the instance profile, and the ec2-instance-connect package (preinstalled on AL2023 standard and Ubuntu 20.04+).
On the runner role — ssm:StartSession, ssm:DescribeInstanceInformation, ssm:DescribeSessions, ssm:TerminateSession and ec2-instance-connect:SendSSHPublicKey. The ready-to-paste policy, plus the mistakes that cause most failures, are in docs/IAM.md. 👀 The big one: scoping ssm:TerminateSession with ${aws:username} silently does not work under OIDC federation — use a tag condition instead.
⚠️ Caveats
Sharp edges, up front:
- 🔇 No command-level audit. The tunnel is opaque to AWS, so session logging captures nothing readable and CloudTrail notes only that a session opened. If you need a record of what ran, reach for SSM Run Command instead.
- 🐌 It's a relay, not a pipe. Tens of MB move fine; a multi-gig artifact is miserable. Ship those through S3.
- 🔓 You still own
sshd. The door moved, it didn't disappear — keep patching it. - ⏱️ Keys expire after a minute. Only the first connection has to beat the clock (multiplexing carries the rest, for an hour of
ControlPersist), so put the action right before the steps that use it. Long gaps in the pipeline? Pass your ownprivate-key. - 🏷️ Give overlapping jobs their own
host-alias. Keys, sockets and known_hosts files are named per run, but the config block is keyed on the alias — so two jobs sharing an alias and aHOMEon one self-hosted runner will tread on each other. - 🐧 Linux only, runner and instance both.
Session cleanup, happily, is not on this list: each run stamps its tunnels with a unique --reason marker and the post step terminates only those, so concurrent jobs sharing a role, a runner and an instance never cut off each other. The long version of all of this lives in docs/Caveats.md.
🎬 Wrapping up
Long-lived SSH keys in CI secrets and IP-whitelisted port 22 are habits from before Session Manager existed. Swapping them out costs you one step in a workflow file and an IAM policy — and your deploy commands don't change at all.
🔗 Links
- 📦 The action: github.com/ankurk91/setup-ssh-over-ssm-action
- 🔐 IAM policies and gotchas
- ⚠️ Full caveats
- 📘 AWS: allow SSH connections through Session Manager
- 🔌 AWS: install the Session Manager plugin
- 🗝️ AWS API:
SendSSHPublicKey - 🧰
install-aws-cli-action·install-session-manager-plugin-action
⭐ If this saved you from writing another security-group-updating cron job, star the repo — it's the signal that tells me which parts to keep building, and it helps the next person find it instead of pasting a private key into a secret.
Issues and PRs welcome. 🙌 Closed port 22 already? Tell me how it went in the comments. 💬
