The Orchestration Epiphany
I joined the Kubernetes team in January 2015, and one of my first memories was a late-night chat with Tim Hockin. We were discussing the opportunity in front of us, and he told me about his first experience with Borg, Google's internal orchestration system.
"I remember when I spun up 25,000 jobs on Borg as easily as I spun up a single process on my own machine," he said. "That was the epiphany for him, and his question to me was, 'How do we give that feeling to everyone with Kubernetes?'"
I had certainly never spun up 25,000 jobs; I'd probably never spun up 2.5 jobs simultaneously. But a system that could handle something like that without you knowing or caring about where, when, or how those machines were available? That was truly revolutionary.
The thing is, most people think that revolution was about containers. It wasn't. It was about orchestration, and more specifically, about a fundamental shift in how we think about systems.
The Mental Model Flip
For decades, the fundamental question of infrastructure was: "Where does my application run?"
The answer evolved—from a specific bare-metal server, to a VM in a rack, to a container on a host—but the question remained the same. We were still thinking in terms of location, managing a fleet of individual, addressable things.
Orchestration proposed a radical new question: "What does my application need?"
That shift from "where" to "what" changed everything.
Before orchestration, every conversation sounded like this:
- "The API server needs to be on the machine with the SSD"
- "Make sure the batch job runs on the box with 64GB of RAM"
- "Don't put the database on the same host as the web server"
After orchestration, those conversations disappeared. Not because the constraints went away, but because we learned to express them differently:
- "This service needs fast storage"
- "This job requires 32GB of memory"
- "These workloads shouldn't share a failure domain"
Same requirements. Completely different mental model.
What Orchestration Actually Delivers
Orchestration isn't "a bunch of YAML to run containers." It's a contract between you and the system:
-
Intent over Instructions: You write "maintain five replicas in two regions" instead of a script to run docker on
node-42. The system figures out the implementation. - Automatic Failure Recovery: Hardware dies, pods respawn elsewhere, the service IP never changes. You've moved from managing individual failures to defining system-wide resilience.
- Bin-Packing at Planet Scale: The scheduler plays Tetris with your workloads so you don't pay for empty rectangles. An orchestrator sees the entire cluster as a single pool of resources and optimizes accordingly.
- Declarative Scaling: You raise a number in a config file; capacity appears. No more SSH scripts or manual load balancer updates.
Once you internalize this, the mental model flips from places to needs. You stop being a server administrator and start being a system architect.
The Single Cluster Trap
But here's where most teams get stuck. They master orchestration within a single cluster and think they're done. They can deploy, scale, and heal applications beautifully—as long as everything stays in one place.
The moment they need to run workloads across multiple clusters, regions, or clouds, they're back to thinking like it's 2010. I've seen teams spend months trying to deploy a simple web app consistently across AWS, GCP, and their on-premises environment. They'll have three different YAML files, three different CI/CD pipelines, and three different sets of monitoring dashboards.
That's not orchestration. That's just automation with extra steps.
Data: Still Stuck in the Stone Age
Here's the uncomfortable truth: while we've revolutionized how we think about compute, data is still managed like it's 1995.
Walk into any company and ask them how they handle data movement. I guarantee you'll hear some variation of:
- "We copy everything to the data lake first"
- "We have a nightly batch job that syncs data between systems"
- "We use Kafka to stream data to our analytics cluster"
- "We're migrating everything to Snowflake"
Sound familiar? That's exactly how we used to talk about compute before orchestration. "My app needs to run on the machine with the data." "I need to copy this file to the server that has the GPU."
We solved this for compute by building systems that could intelligently place workloads based on requirements and constraints. But for data, we're still doing the digital equivalent of carrying buckets of water from the well to the house.
The Mental Model Shift We Need
Imagine if you could think about data the same way you think about compute in a modern orchestrator:
Instead of: "Copy this 10TB dataset from our edge devices to AWS, then run the analysis" You'd say: "I need to run this analysis on the latest sensor data, with sub-second latency, keeping raw data in the originating region for compliance."
Instead of: "Set up a pipeline to replicate customer data from Salesforce to our analytics cluster" You'd say: "I need real-time customer insights, with GDPR compliance, accessible from our dashboard."
Instead of: "Let's set up a special sanitization and provenance pipeline in an isolated region so that we can respect our regulations and governance." You'd say: "Let's run our data transformation pipelines in the regulatory region in which they are created, with each one picking up the specific requirements of their region without moving the data."
The orchestrator would figure out where to run the compute, how to access the data, and how to handle failures, scaling, and compliance requirements. Just like container orchestrators do for your applications today.
Here's what that might look like:
analysisJob:
source: "edge://factory/*/temperature"
policy:
residency: "EU"
latency: "<5s"
transform: "docker.io/acme/quality-check:2.1"
output: "s3://analytics/reports/daily"
No scp scripts, no overnight copy jobs, no wondering which region owns the truth. The system decides where the compute should run, how privacy constraints dictate placement, and what happens if the job fails.
Why This Matters Now
The gap between compute orchestration and data orchestration is creating real business problems. Companies that can deploy applications globally in minutes are still spending weeks moving data around. Teams that can scale web services automatically are still manually configuring data pipelines.
This isn't just a technical problem—it's a competitive disadvantage. The companies that figure out data orchestration first are going to have the same advantage over their competitors that early Kubernetes adopters had over teams still manually managing servers.
Where Expanso Fits
At Expanso, we're building the data equivalent of what Kubernetes did for compute. You declare what you need: "I want to run this analysis on customer data, with these privacy constraints, in under 30 minutes."
Our system figures out where the data is, schedules the compute appropriately, and handles all the networking, security, and compliance requirements.
Under the hood, it might spin up a VM, light up a container, or fire a WASM function. You won't notice. That's the point. Abstraction wins when it solves a real headache and then gets out of the way.
Call to Action
If you're running workloads across multiple environments, you already understand the power of orchestration. You know what it feels like to declare intent and let the system handle implementation.
Now ask yourself: why are you still thinking about data the way we used to think about compute? Why are you manually moving datasets when you could be orchestrating data access the same way you orchestrate application deployment?
The future belongs to teams that apply the orchestration mindset to everything—not just compute, but data, networking, security, and compliance. The teams that figure this out first are going to have an enormous advantage.
What's your experience with data orchestration? What clicked for you? What's still missing? I'd love to hear your thoughts.
Series RoadmapThis piece continues our series examining Kubernetes at ten: where it's excelled, where it's stalled, and how we bridge the gap between compute and data in a truly distributed world.
Originally published at Why Single Machines (and Single Clusters) Are Not Enough: The Orchestration Mindset.
