MSK custom domain names shipped 17 August 2026, and the NLB behind them does not scale with your brokers

Summary. Amazon MSK added cluster-level custom domain names for MSK Provisioned clusters on 17 August 2026, on both ZooKeeper and KRaft metadata modes, in every Region where MSK Provisioned runs, at no additional charge. The AWS What's New post says the configuration "persists through scaling operations." The MSK Developer Guide says something the announcement does not: "The networking layer doesn't scale automatically." Both statements are accurate, and the gap between them is a production outage waiting for the next broker you add. MSK resolves {broker_id} for a new broker and advertises its custom address within a rolling restart. The Network Load Balancer listener, the target group and the DNS record for that broker are yours to create by hand, and until you do, clients that refresh metadata get an address that resolves to nothing. A Network Load Balancer costs $0.0225 per hour in US East (N. Virginia) and $0.0239 per hour in Asia Pacific (Mumbai), plus $0.006 per LCU-hour in both, and a general-purpose AWS Private CA to sign the certificate is $400 per month.

What actually shipped

The feature is one Apache Kafka configuration property, custom.advertised.listeners, added to an Amazon MSK configuration and applied with UpdateClusterConfiguration. Its format, from the developer guide:

custom.advertised.listeners=LISTENER_NAME://hostname-pattern:port+{broker_id}

On a three-broker IAM cluster, the guide's own example is:

custom.advertised.listeners=CLIENT_IAM://b-{broker_id}.example.com:9000+{broker_id}

which resolves to b-1.example.com:9001, b-2.example.com:9002 and b-3.example.com:9003. The {broker_id} template variable is required and it must appear in the port, not only in the hostname, so that every broker lands on a unique address. The + operator adds the broker ID to the base port: 9000 + 10 = 9010 for broker 10.

Before 17 August 2026, per the AWS announcement, customers configured custom domain names manually on each broker, and on KRaft-based clusters they could not configure them at all. That is the real change. One property at cluster level now replaces per-broker work, and KRaft clusters get the capability for the first time.

What did not change is everything below the cluster.

The sentence the announcement leaves out

The AWS What's New entry, dated 17 August 2026, states that the configuration "persists through scaling operations and works identically on both ZooKeeper and KRaft-based clusters." Read on its own, that reads like the whole problem is solved.

The developer guide's "Scaling and broker replacement" section agrees on the cluster side and then adds a note:

Add the corresponding Network Load Balancer listener, target group, and DNS record for any new broker. The networking layer doesn't scale automatically.

The troubleshooting page repeats it in the context of a failed update: "If the failure followed a scaling operation, add the corresponding Network Load Balancer listener, target group, and DNS record for any new broker."

So the accurate reading is narrower than the announcement suggests. MSK persists the pattern. It resolves {broker_id} for a broker that did not exist when you wrote the configuration, and it advertises b-7.example.com:9007 to every client that refreshes metadata. Nothing in AWS creates the port 9007 listener on your Network Load Balancer, the target group behind it, or the b-7 DNS record. Scale a cluster from six brokers to nine on a Friday afternoon and three brokers start advertising addresses that do not exist.

The same asymmetry applies to automated healing. The guide says MSK applies the configuration to a replaced broker "with no manual steps required on the cluster side." The qualifier "on the cluster side" is carrying the weight of the sentence.

How it breaks, precisely

The developer guide is blunt about the failure mode, and it puts the warning in two separate places. "When you apply custom.advertised.listeners, your custom domain name replaces the default addresses for the overridden listener." And: "If the networking and trust layer isn't in place, resolvable, reachable, and trusted from the client, the client can't reconnect. This is true even if the client was connected moments earlier."

That last clause matters more than it looks. Kafka clients hold a connection and periodically refresh metadata. A client connected happily to b-7.abcdef.c2.kafka.us-east-1.amazonaws.com:9098 will, at its next metadata refresh, be handed b-7.example.com:9007 instead. If that name does not resolve, or resolves to an NLB with no listener on 9007, or the NLB presents a certificate that does not cover the hostname, the client drops out. It does not fall back to the AWS-generated address, because the custom name has replaced it.

Three checks the guide lists for exactly this situation:

• The NLB listener, target group and security group rule exist for the broker and port the client is using, and the NLB is reachable from the client's network.
• The custom domain resolves to the NLB from every network where Kafka clients run.
• The NLB's TLS certificate covers the resolved broker hostname through the common name or a subject alternative name, and the client trusts the CA, including root and intermediate for a private CA.

The third one is the one teams forget on scale-out. A wildcard certificate for *.example.com covers b-7.example.com for free. A certificate with three explicit subject alternative names for brokers 1 to 3 does not.

What it costs, and where the "no additional cost" ends

MSK charges nothing for the feature itself. The announcement says so, and the developer guide adds no pricing note. The bill lives entirely in the networking layer you have to build.

Read from the AWS Price List API on 22 August 2026 (offer AWSELB, version 20260818181726):

LCU consumption on a Kafka path is not trivial, because NLB LCUs bill on new connections, active connections, processed bytes and rule evaluations, and a busy producer fleet drives all four. Budget the $16 to $18 per month as the floor, not the estimate.

The certificate is where the number gets interesting. If your Kafka clients sit inside your own networks and you sign the custom domain with AWS Private CA, the Price List API for AWSCertificateManager in US East (N. Virginia), read the same day, gives:

A feature that costs nothing therefore costs $416 a month in US East (N. Virginia) if you stand up a general-purpose private CA and one NLB for it, before a single LCU. Short-lived certificate mode brings the CA down to $50 a month and the per-certificate price to $0.058, which suits an automated rotation pipeline and suits nothing else. The real cost here is the networking and PKI you now own, not the Kafka change.

The validation rules that will reject your first attempt

MSK validates the configuration synchronously, so a malformed value never reaches a broker. It returns HTTP 400 and refuses. Two error shapes are documented, and they fire at different points in the workflow.

The format error returns invalidParameter=serverproperties with the message Invalid custom.advertised.listeners format. Expected: LISTENER_NAME://host:port+{broker_id} (comma-separated for multiple). This is a CreateConfiguration problem: the {broker_id} variable is missing from the port, or the separator is wrong.

The listener error returns invalidParameter=configurationInfo with a message of the form Custom advertised listener(s) [CLIENT_SECURE] are not bound on this cluster. Valid client listeners: [CLIENT_IAM]. A broker cannot advertise a listener it does not bind. The guide is explicit that this one "occurs during UpdateClusterConfiguration, not during CreateConfiguration" — you can create a configuration that will never apply to your cluster and only find out at apply time.

Only client listeners are accepted: CLIENT, CLIENT_SECURE, CLIENT_SECURE_PUBLIC, CLIENT_SASL_SCRAM, CLIENT_SASL_SCRAM_PUBLIC, CLIENT_IAM and CLIENT_IAM_PUBLIC. The internal REPLICATION and CONTROLLER listeners are rejected at validation, which closes off any idea of putting inter-broker traffic behind the same custom domain.

Two mechanical details from the setup page that cost time on the first run. Pass the properties file with fileb://, not file://, so the CLI reads it as bytes and base64-encodes it; passing the value inline is fragile because of the {broker_id} braces. And leave {broker_id} literal in the file, because MSK resolves it per broker at apply time.

Quotas and ports worth checking before you start

Two account quotas cap how much of this you can do. MSK allows 100 configurations per account and 50 configuration revisions per account, on both Standard and Express broker tables in the limits page. The revision cap is the tighter of the two for anyone iterating on listener patterns, because every corrected attempt burns a revision. Configurations per account can be raised through the Service Quotas console; the limits page lists no such note against revisions.

The base ports you are overriding, from the MSK port information page: 9092 plaintext, 9094 TLS from within AWS, 9194 TLS public, 9096 SASL/SCRAM, 9196 SASL/SCRAM public, 9098 IAM, 9198 IAM public, and for IPv6 network type 20092, 20094, 20096 and 20098. Choose a custom base port that will not collide with these after {broker_id} is added, and remember the arithmetic: a base of 9090 on a cluster that grows past broker 4 walks into 9094.

How to tell whether this is you

You are exposed if all three are true: you run MSK Provisioned, you have applied or plan to apply custom.advertised.listeners, and your cluster count changes without a human in the loop. That last condition covers more teams than it sounds like. Auto-healing replaces a broker without asking. Storage or broker scaling initiated from a runbook, a Terraform apply or a CDK deploy adds brokers without touching your NLB.

The rollout behaviour gives you one safety net and one trap. MSK applies the configuration through a rolling restart, one broker at a time, and you track it with describe-cluster-operation-v2 until it reports UPDATE_COMPLETE. If it reports UPDATE_FAILED, the guide says the rollout halts at that broker and the remaining brokers keep their previous configuration, so a bad configuration cannot take out the whole cluster in one pass. That is the net. The trap is that a scale-out does not run this workflow at all — there is no UPDATE_FAILED to catch, because nothing failed. MSK did exactly what it was asked.

If you need to back out, the documented path is to remove the custom.advertised.listeners property from the configuration and apply the updated configuration with UpdateClusterConfiguration. Brokers revert to their original addresses through another rolling restart. The guide adds the obvious precondition that is easy to violate in a locked-down VPC: make sure clients can reach the original MSK-generated addresses before you remove the custom domain.

What to do about it

Wire the networking into the same change that scales the cluster, rather than treating it as a follow-up. If broker count is managed in Terraform or CDK, the NLB listener, target group and Route 53 record should be generated from the same broker-count variable, so there is no state in which MSK knows about broker 7 and DNS does not. Use a wildcard certificate for the broker hostname pattern instead of enumerated subject alternative names, so certificate coverage is not a second thing to remember.

Then add an alarm that compares broker count to listener count. The failure is silent from the MSK side; the cluster is healthy, the operation succeeded, and only your clients know anything is wrong.

India-specific considerations

Mumbai (ap-south-1) pays $0.0239 per NLB-hour against $0.0225 in US East (N. Virginia), a 6.2% premium on the fixed component, with the LCU rate identical at $0.006. On a single load balancer that is about $1 a month, so it should not drive a Region choice. The Private CA charge is the line to model, and teams handling personal data under the Digital Personal Data Protection Act 2023 should note that routing Kafka client traffic through an NLB with a private CA keeps the trust chain inside infrastructure you control, which is a cleaner story for a data-flow record than a public endpoint would be.

What is still unknown

The developer guide points at the AWS Big Data Blog walkthrough for the Network Load Balancer, Route 53 and AWS Certificate Manager setup rather than documenting it, so the reference architecture for the networking layer sits outside the service documentation. AWS has published no guidance on whether a future release will manage NLB targets on the customer's behalf, and the limits page carries no quota specific to custom.advertised.listeners itself, so how many distinct listener-to-domain mappings a single cluster will accept in practice is undocumented beyond the rule that each listener maps to one domain.

FAQ

What did Amazon MSK actually ship on 17 August 2026?

Cluster-level custom domain names for MSK Provisioned clusters, set through the custom.advertised.listeners configuration property. It works on both ZooKeeper and KRaft metadata modes, applies to all new and existing MSK Provisioned clusters in every Region where the service runs, and AWS charges nothing extra for it.

Does the configuration really persist when I add brokers?

The MSK side does. AWS resolves the {broker_id} template for a new or replaced broker and advertises its custom address automatically. The developer guide adds that the networking layer does not scale automatically, so the Network Load Balancer listener, target group and DNS record for that broker remain a manual step you must perform.

What happens to connected clients if the DNS record is missing?

They drop. The custom domain name replaces the default AWS-generated address for the overridden listener, so a client that refreshes metadata receives the custom hostname and has no fallback. The guide states this applies even to a client that was connected moments earlier, if the name cannot be resolved or reached.

Which listeners can I put behind a custom domain?

Only client listeners: CLIENT, CLIENT_SECURE, CLIENT_SECURE_PUBLIC, CLIENT_SASL_SCRAM, CLIENT_SASL_SCRAM_PUBLIC, CLIENT_IAM and CLIENT_IAM_PUBLIC. The internal REPLICATION and CONTROLLER listeners are rejected during validation. The listener must also already be bound on your cluster, or the apply fails with an HTTP 400 error naming the valid listeners.

What does the supporting infrastructure cost?

A Network Load Balancer is $0.0225 per hour in US East (N. Virginia) and $0.0239 in Asia Pacific (Mumbai), both with LCUs at $0.006 per hour, which is roughly $16 to $18 a month before traffic. A general-purpose AWS Private CA adds $400 a month, or $50 a month in short-lived certificate mode.

Why does my configuration fail with an HTTP 400 error?

Two documented causes. A format error returns invalidParameter=serverproperties when {broker_id} is missing from the port or listeners are not comma-separated. A listener error returns invalidParameter=configurationInfo when the named listener is not bound on the cluster, and that one fires at update time rather than at configuration creation.

How many configurations and revisions do I get?

Amazon MSK allows 100 configurations per account and 50 configuration revisions per account, on both Standard and Express broker clusters. The configuration quota can be raised through the Service Quotas console. The limits page lists no increase path against the revision quota, which matters if you iterate on listener patterns.

How do I remove a custom domain safely?

Delete the custom.advertised.listeners property from the MSK configuration and apply the updated configuration with UpdateClusterConfiguration. Brokers revert to the original AWS-generated addresses through a rolling restart. Confirm first that every client can reach those original addresses, because otherwise the rollback disconnects the clients it was meant to rescue.

How eCorpIT can help

We run data engineering services and DevOps and platform engineering for teams whose streaming infrastructure has outgrown a single runbook, and the scale-out gap described here is the kind of thing we wire into infrastructure code rather than a checklist. If you are applying custom domain names to an MSK cluster whose broker count changes automatically, we can review the Terraform or CDK path so the load balancer and DNS follow the cluster. Talk to us through /contact-us/.

Related reading on AWS documentation gaps and cloud cost: our analysis of the EKS Argo CD capability configuration conflict, the AWS Glue 6.0 price cut and its version-gated Iceberg v3 support, and the pillar on cutting cloud spend for Indian teams.

References

1. Amazon MSK now supports configuring custom domain names for MSK Provisioned clusters — AWS What's New, 17 August 2026.
2. Configure custom domain names for your Amazon MSK cluster — Amazon MSK Developer Guide.
3. Set up a custom domain name end to end — Amazon MSK Developer Guide, including the "Scaling and broker replacement" note.
4. Troubleshoot your Amazon MSK cluster — custom domain name configuration errors.
5. Amazon MSK quotas — configurations and configuration revisions per account.
6. Port information — Amazon MSK Developer Guide.
7. UpdateClusterConfiguration — Amazon MSK API Reference.
8. Configure a custom domain name for your Amazon MSK cluster — AWS Big Data Blog, the networking walkthrough the developer guide defers to.
9. Elastic Load Balancing pricing — Network Load Balancer hourly and LCU rates, cross-checked against the AWS Price List API offer AWSELB, version 20260818181726, on 22 August 2026.
10. AWS Private CA pricing — cross-checked against the AWS Price List API offer AWSCertificateManager for US East (N. Virginia) on 22 August 2026.
11. Amazon MSK pricing — AWS.
12. Apache Kafka documentation — broker configuration, including advertised listeners.

Last updated: 22 August 2026.