Running Kubernetes in an Air-Gapped Environment With Spinifex

Running Kubernetes in an air-gapped environment is harder than it looks. Kubernetes pulls container images from registries. Control plane components phone home during bootstrap. Node configuration tooling assumes it can reach package repositories. The ecosystem is built around internet connectivity, and peeling that assumption out of a real deployment is tedious enough that most teams avoid it or work around it with expensive and brittle proxy setups.

Spinifex installs without internet access, using a release tarball and pre-staged packages transferred on USB media, and it provides an AWS-compatible EKS implementation so you get managed Kubernetes on hardware that never leaves your network.

What "air-gapped" actually means here

There are different degrees of disconnection, so here is what Spinifex supports and where the constraints are.

The Spinifex platform, covering the AWS gateway, compute, block storage, object storage, networking, and IAM, installs and runs with no internet connectivity at all. Nothing in the install process reaches out to the internet once you've transferred the release tarball and dependencies to the target machine. The --no-telemetry flag on spx admin init prevents the one-shot install telemetry beacon, suppressing even that initial call.

EKS clusters have one constraint: the control plane needs egress during bootstrap. The EKS guide's troubleshooting notes document this as expected behaviour for the current release. Once the cluster is active, your workloads can run in a LAN with no external routes as long as container images are available locally, either pre-pulled or served from a Spinifex ECR instance on the same network. The platform itself doesn't require connectivity for ongoing operation.

Preparing the offline installer

The air-gapped install process works in two phases: prepare everything you need on a connected machine, then transfer and install on the target.

On a machine with internet access, download the Spinifex release tarball, the install script, and the SHA-256 checksum:

ARCH=amd64   # or arm64
TAG=$(curl -fsSL https://api.github.com/repos/mulgadc/spinifex/releases/latest \
  | grep '"tag_name"' | cut -d'"' -f4)

BASE="https://github.com/mulgadc/spinifex/releases/download/${TAG}"
curl -fsSLO "${BASE}/spinifex-${TAG}-linux-${ARCH}.tar.gz"
curl -fsSLO "${BASE}/spinifex-${TAG}-linux-${ARCH}.tar.gz.sha256"
curl -fsSLO "${BASE}/setup.sh"

sha256sum -c "spinifex-${TAG}-linux-${ARCH}.tar.gz.sha256"

Pre-download the APT packages the installer needs. Using --download-only writes the .deb files to the apt cache without installing them, so the connected machine is unaffected:

sudo apt update
sudo apt install --download-only -y \
  nbdkit nbdkit-plugin-dev pkg-config \
  qemu-system-x86 qemu-utils \
  ovmf qemu-efi-aarch64 \
  libvirt-daemon-system libvirt-clients libvirt-dev \
  ovn-central ovn-host openvswitch-switch \
  dhcpcd-base make gcc jq curl iproute2 netcat-openbsd \
  wget unzip xz-utils file

Download the AWS CLI v2 installer and the cloud image you'll use as the guest OS for virtual machines:

curl -fsSL "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" \
  -o awscliv2.zip

mkdir -p images
curl -fsSL "https://cloud.debian.org/images/cloud/trixie/latest/debian-13-genericcloud-amd64.qcow2" \
  -o images/debian-13-amd64.qcow2

Stage everything onto the transfer media:

mkdir -p /media/spinifex-deploy/{tarball,apt-packages,aws,images}
cp spinifex-${TAG}-linux-${ARCH}.tar.gz /media/spinifex-deploy/tarball/
cp setup.sh /media/spinifex-deploy/
cp /var/cache/apt/archives/*.deb /media/spinifex-deploy/apt-packages/
cp awscliv2.zip /media/spinifex-deploy/aws/
cp images/*.qcow2 /media/spinifex-deploy/images/

Installing on the air-gapped target

On the target server, with no internet connection, mount the transfer media and work through the install sequence.

Install the APT packages from the local cache:

sudo mount /dev/sdb1 /mnt/usb

sudo dpkg -i /mnt/usb/apt-packages/*.deb
sudo apt-get install -f --no-download

The --no-download flag tells apt to resolve any outstanding dependencies using only what's already in the local cache. If a dependency is missing, you'll need to add it to the --download-only step on the connected machine and re-stage.

Install the AWS CLI and run the Spinifex installer, pointing it at the local tarball and suppressing the package and AWS download steps:

cd /tmp && unzip /mnt/usb/aws/awscliv2.zip && sudo ./aws/install

INSTALL_SPINIFEX_TARBALL=/mnt/usb/tarball/spinifex-*-linux-*.tar.gz \
INSTALL_SPINIFEX_SKIP_APT=1 \
INSTALL_SPINIFEX_SKIP_AWS=1 \
bash /mnt/usb/setup.sh

Run setup.sh without sudo. The script handles privilege escalation internally and ends by launching a newgrp spinifex subshell so your current shell picks up the spinifex group membership, which is required for the AWS CLI to read the TLS certificates in /etc/spinifex/.

Configure OVN networking, initialise without telemetry, and start the services:

sudo /usr/local/share/spinifex/setup-ovn.sh --management

sudo spx admin init --node node1 --nodes 1 --no-telemetry

sudo systemctl start spinifex.target

Import the cloud image from the USB media:

sudo spx admin images import --file /mnt/usb/images/debian-13-amd64.qcow2 \
  --distro debian --version 13 --arch x86_64 --boot-mode uefi

Verify the install worked:

export AWS_PROFILE=spinifex-<nodename>
aws ec2 describe-instance-types
aws ec2 describe-images

Both calls returning data confirms the AWS gateway is up and the image is registered. At this point you have a fully functional AWS-compatible cloud platform running on isolated hardware.

Running EKS clusters on the disconnected platform

With Spinifex running, you can create EKS clusters using the standard AWS CLI workflow. The spx admin init command creates the spinifex-<nodename> AWS profile automatically, pointing all API calls at the local Spinifex gateway on port 9999.

Before creating a cluster, confirm the eks-node image is registered, because Spinifex blocks cluster creation until it's present:

aws ec2 describe-images \
  --filters 'Name=tag:spinifex:managed-by,Values=eks' \
  --query 'Images[].[ImageId,Name]' --output text

If the EKS node image isn't in your catalogue yet, you'll need to stage it on the transfer media along with the other cloud images and import it with spx admin images import before proceeding.

With the image in place, create the cluster:

export AWS_PROFILE=spinifex-<nodename>

aws eks create-cluster \
  --name demo \
  --role-arn arn:aws:iam::000000000000:role/eks-cluster-role \
  --resources-vpc-config subnetIds=subnet-aaaa,subnet-bbbb,endpointPublicAccess=true \
  --access-config authenticationMode=API \
  --kubernetes-version 1.32

The authenticationMode=API flag is required because Spinifex implements the access-entry authentication model rather than the older aws-auth ConfigMap approach.

During cluster bootstrap the control plane needs egress, so ensure the subnet the control plane lands in has an internet gateway route, or confirm the route is in place temporarily. Once the cluster reaches the ACTIVE state, the control plane doesn't need ongoing connectivity for its own operation.

aws eks wait cluster-active --name demo

aws eks create-nodegroup \
  --cluster-name demo --nodegroup-name default \
  --node-role-arn arn:aws:iam::000000000000:role/eks-node-role \
  --subnets subnet-aaaa subnet-bbbb \
  --scaling-config minSize=1,maxSize=2,desiredSize=1 \
  --instance-types t3.medium

aws eks update-kubeconfig --name demo
kubectl get nodes

Running workloads with no external connectivity

Once the cluster is active, the blocking question for disconnected environments is image availability. Kubernetes pods pull images when they start, and without internet access the nodes need to reach a local image source.

Spinifex includes an ECR-compatible container registry for hosting images on the same network as the cluster. Push images to a local Spinifex ECR instance, configure your pod specs to pull from it, and the entire workload lifecycle from image push through deployment stays within the local network, with no dependency on Docker Hub, GitHub Container Registry, or any external registry.

The practical benefit

The reason to go through this process rather than using a simpler Kubernetes distribution is operational continuity. Engineers who know how to operate EKS on AWS know how to operate EKS on Spinifex, because the API surface, the CLI commands, the Terraform resource types, and the IAM model are identical. AWS tooling and operational knowledge transfers to the disconnected environment without a relearning curve.

For defence, industrial, and classified environments where the security requirement is hardware you own and control, that means a production Kubernetes platform without a separate body of tooling knowledge or a reduced feature set.

Full documentation, including Terraform workbooks for common EKS configurations, is at docs.mulgadc.com. The air-gapped install guide is at docs.mulgadc.com/docs/install-airgapped. Spinifex is open source, written in Go, and AGPL-3.0 licensed, with the full source at GitHub.