Migrating a Terraform Module to Ephemeral Resources Without Breaking Existing Users

Terraform's ephemeral resources are a genuinely useful addition — secrets that are used during an apply but never written to state.

If you're a module maintainer, adopting ephemerals isn't as simple as swapping one block for another. You have existing users. Their state files already have the old resource tracked. Changing to an ephemeral version of the resource will blow away their managed resource.

To illustrate the issue, we'll pretend to be updating a module that generates a TLS private key and stores it in a Vault KV secret. This post documents the problem and a solution with an extra tid-bit that I learned.

The Setup

The original module was straightforward:

resource "tls_private_key" "legacy" {
  algorithm = "RSA"
  rsa_bits  = 4096
}

resource "vault_kv_secret_v2" "legacy" {
  mount     = "kvv2"
  name      = "mytls"
  data_json = jsonencode({ private_key = tls_private_key.legacy.private_key_pem })
}

The private key ends up in state. Vault KV ends up with a copy. Works fine, but the key is visible to anyone with state access.

The goal: keep this working for existing users, while letting new users opt into ephemeral key generation — where the key is used to write to Vault but is never persisted to state.

For solution TL;DR see The Complete Solution below

The Naive Approach (and Why It Fails)

The obvious move is a variable toggle with a local to abstract the logic away from the resource:

variable "use_ephemeral_key" {
  type    = bool
  default = false
}

resource "tls_private_key" "legacy" {
  count     = var.use_ephemeral_key ? 0 : 1
  algorithm = "RSA"
  rsa_bits  = 4096
}

ephemeral "tls_private_key" "ephemeral" {
  count     = var.use_ephemeral_key ? 1 : 0
  algorithm = "RSA"
  rsa_bits  = 4096
}

locals {
  private_key_pem = (
    var.use_ephemeral_key
    ? ephemeral.tls_private_key.ephemeral[0].private_key_pem
    : tls_private_key.legacy[0].private_key_pem
  )
}

resource "vault_kv_secret_v2" "legacy" {
  count     = var.use_ephemeral_key ? 0 : 1
  mount     = "kvv2"
  name      = "mytls"
  data_json = jsonencode({ private_key = local.private_key_pem })  
}

Running this produces:

│ Error: Invalid use of ephemeral value
│
│   with vault_kv_secret_v2.legacy,
│   on updated.tf line 34, in resource "vault_kv_secret_v2" "legacy":
│   34:     data_json = jsonencode({ private_key = local.private_key_pem })
│
│ Ephemeral values are not valid for "data_json", because it is not a
│ write-only attribute and must be persisted to state.

Why This Happens

The fundamental rule is: ephemeral values can only flow into write-only attributes. data_json must be persisted to state, so it can never accept an ephemeral value. That's true whether the ephemeral value arrives directly or through a local.

The shared local makes this worse in a subtle way. Terraform's ephemeral taint propagation is static, not runtime. Because one branch of the local.private_key_pem conditional references an ephemeral value, the entire local is marked ephemeral — regardless of which branch actually executes at runtime. So even if you reasoned "when use_ephemeral_key = false, the local resolves to a non-ephemeral value," Terraform doesn't do branch-aware type narrowing. The local is tainted, and the error fires before it even reaches data_json.

The fix is to never route ephemeral and non-ephemeral values through the same expression. vault_kv_secret_v2 has two relevant attributes: data_json (regular, persists to state) and data_json_wo (write-only, accepts ephemeral values). Use them conditionally on the same resource, keeping each attribute's expression isolated to its own value source:

resource "vault_kv_secret_v2" "legacy" {
  mount        = "kvv2"
  name         = "mytls"
  data_json    = var.use_ephemeral_key ? null : jsonencode({ private_key = one(tls_private_key.legacy[*].private_key_pem) })
  data_json_wo = var.use_ephemeral_key ? jsonencode({ private_key = one(ephemeral.tls_private_key.ephemeral[*].private_key_pem) }) : null
}

Why one() instead of [0]: Both key resources use count — only one is active at a time. Referencing resource[0] when count = 0 errors at plan time even when the ternary condition would skip that branch, because Terraform resolves resource instance addresses before evaluating expressions. The splat + one() pattern returns null for an empty collection instead of erroring:

one(tls_private_key.legacy[*].private_key_pem)  # null when count = 0, actual value when count = 1

The outer ternary ensures that null never reaches jsonencode, so neither attribute ends up with an unexpected {"private_key":null} payload.

Rule to internalize: ephemeral values can only ever terminate at ephemeral attributes (write-only, or ephemeral such as provider blocks). Don't route them through any expression that also serves a non-write-only path.

The Second Problem: moved Blocks

The original module had no count on either resource. Adding count to tls_private_key.legacy changes its state address:

Terraform treats these as different resources. Without intervention, it will destroy the old address and create a new one — generating a brand new private key in the process. For existing users, that's an unannounced key rotation.

vault_kv_secret_v2.legacy keeps no count, so its address doesn't change and needs no intervention. Only the TLS key resource needs a moved block:

moved {
  from = tls_private_key.legacy
  to   = tls_private_key.legacy[0]
}

This tells Terraform to update the state address in place — no destroy, no create, no key rotation. On the next plan, existing users will see:

# tls_private_key.legacy has moved to tls_private_key.legacy[0]

Leave the moved block in for at least one release cycle so users who haven't applied yet still get a clean migration. It can be removed in a future major version once the old address no longer exists in any user's state.

The Complete Solution

moved {
  from = tls_private_key.legacy
  to   = tls_private_key.legacy[0]
}

variable "use_ephemeral_key" {
  description = "Use ephemeral key generation (key never written to state). New deployments should set true."
  type        = bool
  default     = false
}

# Legacy path — key stored in state
resource "tls_private_key" "legacy" {
  count     = var.use_ephemeral_key ? 0 : 1
  algorithm = "RSA"
  rsa_bits  = 4096
}

# Ephemeral path — key never written to state
ephemeral "tls_private_key" "ephemeral" {
  count     = var.use_ephemeral_key ? 1 : 0
  algorithm = "RSA"
  rsa_bits  = 4096
}

resource "vault_kv_secret_v2" "legacy" {
  mount        = "kvv2"
  name         = "mytls"
  data_json    = var.use_ephemeral_key ? null : jsonencode({ private_key = one(tls_private_key.legacy[*].private_key_pem) })
  data_json_wo = var.use_ephemeral_key ? jsonencode({ private_key = one(ephemeral.tls_private_key.ephemeral[*].private_key_pem) }) : null
}

Versioning Strategy

With this in place, the recommended release approach is:

• Current minor version: ship the toggle with default = false. Existing users are unaffected. New users can opt in with use_ephemeral_key = true.
• Next major version: flip the default to true. Document the change. Users upgrading from the old version need to either set use_ephemeral_key = false explicitly to preserve legacy behavior, or accept that a new key will be generated on their next apply.

The moved blocks handle the state address migration for everyone regardless of which path they're on.