Day 65: An Unread ConfigMap and an Unattached NSG

Day 65 of DevOps, Day 15 of Azure. Both tasks today produced an object that looks configured and enforces nothing. A ConfigMap mounted into a Redis container that Redis never opens, and a network security group with two carefully ordered rules and nothing attached to apply them to.

Both passed. Neither does what its shape suggests.

One Kubernetes task, one Azure task. Deploy Redis with a ConfigMap, then create an NSG with two inbound rules. The tasks come from the KodeKloud Engineer platform.

Task 1: Redis with a ConfigMap

The task: a ConfigMap my-redis-config with the key redis-config carrying a maxmemory setting, and a Deployment redis-deployment running redis:alpine with one replica, port 6379, a CPU request of 1, an emptyDir at /redis-master-data, and the ConfigMap mounted at /redis-master.

Step 1: The ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-redis-config
data:
  redis-config: |
    maxmemory=2mb

Step 2: Generate the Deployment, then fill it in

kubectl create deploy redis-deployment --replicas=1 --image=redis:alpine \
  --dry-run=client -o yaml > deployment.yaml

    spec:
      volumes:
        - name: data
          emptyDir: {}
        - name: redis-config
          configMap:
            name: my-redis-config
      containers:
      - image: redis:alpine
        name: redis-container
        ports:
        - containerPort: 6379
        resources:
          requests:
            cpu: "1"
        volumeMounts:
        - name: data
          mountPath: /redis-master-data
        - name: redis-config
          mountPath: /redis-master

Step 3: Apply and check

kubectl apply -f config.yaml
kubectl apply -f deployment.yaml
kubectl get deploy
kubectl get po

NAME               READY   UP-TO-DATE   AVAILABLE   AGE
redis-deployment   1/1     1            1           21s

NAME                                READY   STATUS    RESTARTS   AGE
redis-deployment-6b4f7dfc48-c629j   1/1     Running   0          33s

Every requirement is met. describe deploy lists the CPU request, both mounts and the ConfigMap volume. describe po shows QoS Class: Burstable, which follows from setting a request and no limits.

The part that does not work

Nothing in this manifest passes the mounted file to Redis.

There is no command and no args, so the container runs the image's entrypoint, which starts redis-server with no configuration file argument. Redis is explicit about what that requires: "Note that to read the configuration file, Redis must be started with the file path as first argument." And about what happens otherwise: "Redis is able to start without a configuration file using a built-in default configuration, however this setup is only recommended for testing and development purposes."

So maxmemory keeps its default, which the docs give as zero: "Set maxmemory to zero to specify that you don't want to limit the memory for the dataset. This is the default behavior for 64-bit systems."

I did not run CONFIG GET maxmemory on this pod, so that is reasoning from three documented facts rather than something I watched. The Kubernetes tutorial for this exact setup publishes the transcript, and it shows maxmemory as "0" and maxmemory-policy as "noeviction" before it recreates the pod with a config-reading command, then "2097152" and "allkeys-lru" afterwards.

There is a second problem waiting behind the first. A ConfigMap volume names each file after its key, so the mount produces /redis-master/redis-config. Anyone who adds a command copied from the tutorial would point Redis at /redis-master/redis.conf, which does not exist here.

And a third. redis.conf directives are space-separated. The value in my ConfigMap reads maxmemory=2mb with an equals sign, which is not the directive syntax. A grader can match that string; Redis cannot parse it.

What the official example does differently

command:
  - redis-server
  - "/redis-master/redis.conf"
volumes:
- name: config
  configMap:
    name: example-redis-config
    items:
    - key: redis-config
      path: redis.conf

Two additions, both necessary. The command hands Redis the path. The items block renames the key to redis.conf so that path exists. Drop either one and the file goes unread.

What the docs recommend

If a container takes a config file, give it the path. The mount is only half of it, and nothing in Kubernetes will tell you the other half is missing, because from the cluster's point of view the volume mounted correctly and the container is running.

Get the units right while you are there. redis.conf's table: "1mb => 1024*1024 bytes", so 2mb is 2097152. 2m would be 2000000, since "1m => 1000000 bytes". The suffix carries the difference between binary and decimal.

Know what allkeys-lru is and is not. The docs define it as "Evict the least recently used (LRU) keys" and call it "a good default option if you have no reason to prefer any others". That is advice, not the default. The default policy is noeviction, so an unconfigured Redis under memory pressure returns errors on writes rather than evicting anything.

Do not reach for CONFIG SET as a fix. It changes the running server and not the file: "modifying the configuration on the fly does not affect the redis.conf and redis-full.conf files, so at the next restart of Redis, the old configuration will be used instead." The command that persists it is CONFIG REWRITE, and on this pod it would fail, because "if the server was started without a configuration file at all, the CONFIG REWRITE will just return an error."

Size the request against the node. A CPU request of 1 is a whole core, since "1 CPU unit is equivalent to 1 physical CPU core". On a single-node lab cluster, that is most of what exists. Worth noting the docs never actually write that 1 equals 1000m; they give the fractional form as 0.1 equaling 100m and leave the rest to arithmetic.

Task 2: An NSG with two rules

The task: create devops-nsg with Allow-HTTP on port 80 and Allow-SSH on port 22, both from 0.0.0.0/0.

Step 1: A failure that described the wrong problem

My first attempt skipped creating the NSG and went straight to the rule:

(RequestDisallowedByPolicy) Resource 'Allow-HTTP' was disallowed by policy.
Reasons: 'Global: This resource location is not allowed. Please use one of the
approved locations: eastus, southcentralus, centralus, westus.'

Nothing in that message is about the real problem, which is that the parent did not exist. The fix was az network nsg create.

I had an explanation for the shape of the error: a security rule is a child resource with no location of its own, so with nothing to inherit from, ARM evaluated it as Global, and the subscription's Azure Policy rejects that.

It fits, and it is not documented. What the documentation supports is narrower. The ARM reference for securityRules lists four top-level properties and no location, while the parent network security group does have one. So the schema omits location for the child. No Microsoft page states that a child inherits its parent's location, and the general guidance says you "must provide a location for each resource". Microsoft's own page on this error attributes it entirely to policy and never mentions child resources or missing parents.

So the lesson is the observation: create the parent first, and a Global policy error on a child is what a missing parent looked like. The mechanism is my reconstruction.

Step 2: Create the NSG, then the rules

az network nsg create -g $RG -n devops-nsg

az network nsg rule create -g $RG --nsg-name devops-nsg -n Allow-HTTP \
  --priority 100 --direction Inbound --access Allow --protocol Tcp \
  --source-address-prefixes 0.0.0.0/0 --destination-port-ranges 80

az network nsg rule create -g $RG --nsg-name devops-nsg -n Allow-SSH \
  --priority 110 --direction Inbound --access Allow --protocol Tcp \
  --source-address-prefixes 0.0.0.0/0 --destination-port-ranges 22

Priority has no default, and the CLI lists it as required. Microsoft: "A number between 100 and 4096. Rules are processed in priority order, with lower numbers processed before higher numbers because lower numbers have higher priority. Once traffic matches a rule, processing stops."

100 and 110 rather than 100 and 101, so a rule can be inserted between them later without renumbering everything.

Tcp rather than *, because * would also open UDP on 80 and 22, and neither HTTP nor SSH uses it.

Step 3: List the rules, including the ones you did not write

az network nsg rule list -g $RG --nsg-name devops-nsg --include-default -o table

Every NSG ships with defaults that custom rules have to beat:

DenyAllInBound at 65500 is why an NSG with no custom rules blocks everything from outside the VNet. The defaults sit outside the 100 to 4096 custom range on purpose, so your rules always run first, and "You can't remove the default rules, but you can override them by creating rules with higher priorities."

They are also in a different place in the API. securityRules holds yours; defaultSecurityRules holds these. A query against the first returns two rules and looks complete.

The source that is not what I thought

"Src": "0.0.0.0/0"

Azure stored exactly what I sent rather than normalising it, which matters for a graded task where a checker may be matching the literal string.

* would have been identical: "0.0.0.0/0 in the Source and Destination columns represents all IP addresses. Clients like the Azure portal, Azure CLI, or PowerShell can use * or Any for this expression."

I had been treating the service tag Internet as a third spelling of the same thing. It is not. Microsoft defines it as "The IP address space that's outside the virtual network and reachable by the public internet." Outside is the operative word: Internet excludes the virtual network's own address space, and 0.0.0.0/0 does not. Substituting one for the other changes which traffic matches.

What the docs recommend

Remember that an unattached NSG enforces nothing. This one is associated with no NIC and no subnet, so its two rules are inert configuration. The task asked only for the NSG and its rules, so I left it that way, and the moment it is attached to anything, Allow-SSH from 0.0.0.0/0 exposes SSH to the whole internet.

Mind the ordering when adding rules later, because this is where Azure differs from AWS in a way that bites. An AWS security group cannot contradict itself: "You can specify allow rules, but not deny rules", and it "Evaluates all rules before deciding whether to allow traffic". An Azure NSG has Deny rules and stops at the first match, so a new Deny at priority 100 silently disables an Allow at 200.

AWS does have an equivalent construct, and it is the network ACL rather than the security group: "Each rule has a number from 1 to 32766. We evaluate the rules in order, starting with the lowest numbered rule... If the traffic matches a rule, the rule is applied and we do not evaluate any additional rules." That is much closer to an NSG, with the difference that network ACLs are stateless.

One correction to something I had written about the API. Azure keeps singular and plural properties for source and port, sourceAddressPrefix and sourceAddressPrefixes, and I had said that exactly one is populated while the other returns an empty array. The REST reference does not say that, and its own sample response omits the plural keys entirely rather than returning []. The empty array is what the Azure CLI renders for an unset list. The practical warning survives and now points at the right layer: a query written against one form breaks when someone adds a second CIDR.

What I learned

Both halves ended with an object that satisfies its requirements and changes nothing. The ConfigMap is mounted at the right path with the right key, and Redis never opens it. The NSG has the right rules in the right order and is attached to nothing.

What makes the first one worse is that every signal available inside Kubernetes says it worked. The volume mounted, the container is running, describe lists everything the task asked for. The only way to find out is to ask Redis what its configuration actually is, which is a step outside the system that was doing the reporting.

So here is the Day 65 question. What in your setup is mounted, loaded, attached or deployed, and has never been asked whether it is actually in effect?

Day 65 down. Thirty-five to go.