Consul service discovery with Nomad and Fabio
the problem
When you run multiple containers of the same image, each gets a random port. Consul solves this - services register themselves and clients query by name instead of host:port.
run Consul
asdf plugin-add consul
asdf install consul 1.19.0
asdf local consul 1.19.0
Or through Nomad using raw_exec:
job "consul" {
datacenters = ["dc1"]
group "consul" {
task "consul" {
driver = "raw_exec"
config {
command = "consul"
args = ["agent", "-dev"]
}
artifact {
source = "https://releases.hashicorp.com/consul/1.19.0/consul_1.19.0_linux_amd64.zip"
}
}
}
}
UI at http://127.0.0.1:8500/ui
register a service
Add a service block to any Nomad task:
task "server" {
driver = "docker"
service {
name = "http-echo"
port = "http"
tags = ["http-echo", "urlprefix-/echo"]
check {
name = "alive"
type = "http"
path = "/"
interval = "10s"
timeout = "2s"
}
}
}
internal load balancing with Fabio
Fabio reads urlprefix-/path tags from Consul and routes traffic automatically.
job "fabio" {
datacenters = ["dc1"]
group "fabio" {
network {
port "lb" { static = 9998 }
port "ui" { static = 9999 }
}
task "fabio" {
driver = "docker"
config {
network_mode = "host"
image = "fabiolb/fabio:1.6.3"
args = ["-proxy.strategy=rr"]
}
resources {
cpu = 200
memory = 128
}
}
}
}
Routes dashboard: http://localhost:9998/routes
Once Fabio is running and your service has urlprefix-/echo, requests to :9999/echo are load balanced across all healthy instances.
# watch round-robin in action
watch -n 0.2 "curl -s http://localhost:9999/echo"
Consul KV for environment variables
Store config in Consul KV and inject it into tasks via the template block:
template {
data = <<EOH
DB_HOST="{{ key "config/db_host" }}"
DB_PASS="{{ key "config/db_pass" }}"
EOH
destination = "secrets/file.env"
env = true
}
Set values at http://127.0.0.1:8500/ui/dc1/kv or via CLI:
consul kv put config/db_host "db01.example.com"
consul kv put config/db_pass "secret"
Originally published at https://bard.sh/posts/consul_service_discovery/
