How Do I Manage Azure Key Vault Secrets Across Development, Staging, and Production?

Why use a separate Key Vault for each environment?

Because the vault, not the secret, is the unit of access control.

Microsoft's Key Vault security guidance recommends one vault per application, region, and environment, with separate vaults for development, preproduction, and production. The reason shows up in the permission model rather than in policy. Role assignments scoped to an individual secret cover read operations only; anything administrative — creating and updating objects, network access control, monitoring — needs a role assigned at the vault level. Firewall rules, private endpoints, and diagnostic settings are per vault as well. Put development and production in one vault and you can't give a developer write access to the development secrets without giving them the same access over production.

Microsoft's ASP.NET Core documentation says it more directly. In the section on secret name prefixes, it warns against using a prefix to "place environmental secrets (for example, Development versus Production secrets) into the same key vault," on the grounds that "different apps and development/production environments should use separate key vaults to isolate app environments for the highest level of security."

So the architecture is settled before you start. What isn't settled is how to live with it.

What does one vault per environment actually cost you?

Adding a single secret turns into three separate operations against three separate resources.

Say a developer needs ThirdParty--ApiKey in all three environments. In the portal that's: open the development vault, create the secret, open the staging vault, create it again, open the production vault, create it a third time, then go back and confirm that all three names match character for character and that each holds the value meant for that environment. Multiply by a few dozen secrets and a handful of applications and vault administration becomes a real line item in the deployment process.

Two failure modes come out of that, and the second is the expensive one.

The first is a typo. ThirdParty--ApiKey in two vaults and Thirdparty--ApiKey in the third resolves to a different configuration key, and the application falls back to whatever default it has rather than failing outright.

The second is drift. A secret gets added to staging during debugging and never makes it to production, or production gets a rotated credential that development doesn't. Nothing in Azure compares vaults for you, the portal has no search across them, and the difference surfaces at deployment time or later. Finding it is a cross-vault search problem rather than anything Key Vault will answer on its own.

How should secret names be kept consistent across environments?

Use exactly the same secret name in every environment, and put the environment in the vault name instead.

The naming rules are tighter than most people expect. A Key Vault object name is 1 to 127 characters, and only 0-9, a-z, A-Z, and -. No colons, no underscores, no periods. That last restriction is why the double-dash convention exists: ASP.NET Core uses a colon to separate a configuration section from its subkey, Key Vault won't accept a colon, so the Key Vault configuration provider replaces -- with a colon when it loads a secret. Database--ConnectionString arrives in configuration as Database:ConnectionString. Arrays follow the same rule with numeric segments, so Logging--Rules--0--Level becomes Logging:Rules:0:Level.

That gives you a secret name that is a property of the application, not of the environment:

Database--ConnectionString
Redis--ConnectionString
Stripe--SecretKey
SendGrid--ApiKey
Storage--ConnectionString

The same five names exist in all three vaults, with three different sets of values. Configuration code stays identical across environments, and a diff of two vaults' secret names is meaningful, because a difference means something is genuinely missing.

Vault names are where the environment goes, and they are the constrained half of the problem. A vault name is 3 to 24 characters, globally unique across all of Azure, alphanumerics and hyphens only, with no consecutive hyphens, because it becomes a DNS name under vault.azure.net. Twenty-four characters doesn't go far once the name has to carry an organization, an application, an environment, and sometimes a region, and still be unique against every other Azure customer:

kc-shop-dev-eus     15 characters
kc-shop-stg-eus     15
kc-shop-prd-eus     15

Decide the scheme before you create the first vault, because a vault can't be renamed and a name you release isn't immediately available again. Soft delete is on by default for new vaults and can't be turned off, the retention period is 7 to 90 days with 90 as the default, it can only be set when the vault is created, and the name can't be reused until that period expires. Delete kc-shop-dev-eus on a default configuration and nobody gets that name back for three months.

How do I add the same secret to development, staging, and production?

Script it, and make the script safe to run twice.

The shape most teams end up with is a name, a set of vaults, and a value per vault:

#!/usr/bin/env bash
set -euo pipefail

secret=ThirdParty--ApiKey

# One file per environment, written with printf so no trailing newline ends up
# inside the secret value, and removed on the way out.
declare -A files=(
  [kc-shop-dev-eus]=./dev.value
  [kc-shop-stg-eus]=./stg.value
  [kc-shop-prd-eus]=./prd.value
)

trap 'rm -f ./dev.value ./stg.value ./prd.value' EXIT

for vault in "${!files[@]}"; do
  present=$(az keyvault secret list --vault-name "$vault" \
    --query "[?name=='$secret'] | length(@)" -o tsv)

  if [[ "$present" != "0" ]]; then
    echo "SKIP $vault ($secret already exists)"
    continue
  fi

  az keyvault secret set --vault-name "$vault" --name "$secret" \
    --file "${files[$vault]}" --encoding utf-8 \
    --content-type "text/plain" --output none

  echo "SET  $vault"
done

Four details in there are worth stealing.

• Values come from files, not --value. They never appear on a command line where they would be visible in shell history, process listings, and pipeline logs. Write those files with printf '%s' rather than echo, because a trailing newline becomes part of the stored value and produces an authentication failure that looks like a wrong credential rather than a formatting mistake.
• The existence check lists rather than shows. Listing needs only the permission to read metadata, and set on a secret that already exists quietly creates a new version rather than failing, so without the check a second run adds versions everywhere.
• --content-type is worth filling in habitually. It's free text, it shows in the portal, and it's the only hint anyone gets about what a secret is for without reading its value.
• --encoding utf-8 is the default, and it's worth knowing that the CLI records the encoding as a file-encoding tag on the secret so az keyvault secret download can reverse it later.

One limit to keep in view if you fan this out across many vaults at once. Key Vault allows 300 CREATE secret transactions per vault per region in any 10 seconds, shared with certificate and key imports, and the subscription-wide ceiling is five times a single vault's limit rather than the sum of them. Writing in parallel to ten vaults in one subscription shares 1,500 creates per 10 seconds, not 3,000, and past that you get HTTP 429 and need to retry with backoff.

This is the operation KickCatch Secrets Manager is built around: you name the secret once, choose the vaults it belongs in, and give each vault its own value. Vaults can be chosen by a naming pattern such as kc-shop-{Environment}-eus rather than picked off a list, so "all three environments" stays a single selection as the vault count grows.

How do I keep secrets synchronized between environments?

Synchronize the names. Manage the values deliberately.

Copying values between environments is almost always the wrong instinct, because the value is the part that is supposed to differ:

What should match is the set of names, and that's worth checking on a schedule rather than at deployment time:

#!/usr/bin/env bash
# Secret names present in one vault but not the other.
set -euo pipefail

names() {
  az keyvault secret list --vault-name "$1" --query "[].name" -o tsv | sort
}

diff <(names kc-shop-stg-eus) <(names kc-shop-prd-eus) \
  | sed -n 's/^< /only in staging:    /p; s/^> /only in production: /p'

az keyvault secret list leaves out certificate-backed secrets unless you pass --include-managed, which is what you want here — every certificate has a secret behind it, and including them adds noise that isn't drift. Run this against staging and production after each release and a missing secret turns up before someone finds it in a log.

When does a changed secret actually reach each environment?

Later than most people assume, and by a different amount in each environment. This is the part of multi-environment management that produces the confusing incidents.

App Service and Functions Key Vault references. A reference without a version pins to the latest value, but the platform caches what it resolved. Microsoft's documentation is specific: "the app is automatically updated and begins using the latest version within 24 hours ... App Service caches the values of the Key Vault references and refetches them every 24 hours. Any configuration change to the app causes an app restart and an immediate refetch of all referenced secrets." To force it without waiting or restarting, post to the refresh endpoint:

az rest --method post --url \
  "https://management.azure.com/<resource-id>/config/configreferences/appsettings/refresh?api-version=2022-03-01"

The .NET Key Vault configuration provider. AddAzureKeyVault doesn't poll by default. AzureKeyVaultConfigurationOptions.ReloadInterval defaults to null, which means configuration isn't reloaded — secrets are read once at startup and the process keeps them until it restarts. If you rotate a credential and the application carries on using the old one, this is usually why. Set a ReloadInterval if you want polling, and remember that each reload lists and reads the vault again, which counts against the transaction limits above.

Your own SDK calls. SecretClient.GetSecretAsync returns the current value every time, so whatever you cache is what you get. Caching is the right call — Microsoft's guidance for secrets recommends it over reading on every request — but the cache lifetime then becomes your rotation delay.

Should Key Vault references be slot settings?

Yes, and this is the one that catches people out during a swap. Microsoft's guidance is blunt about it: "Because you should have separate vaults for each environment, most app settings that use Key Vault references should be marked as slot settings." A setting that isn't a slot setting travels with the slot during a swap, so a staging slot pointing at the staging vault swaps into production still pointing at the staging vault. The application comes up healthy and reads the wrong database.

Taken together, a rotation isn't finished when the new value is in the vault. It's finished when every environment that reads it has picked it up, which means knowing, for each one, whether it polls, caches for a day, or waits for a restart.

How do I provision a new environment quickly?

Treat the secret structure as part of the deployment template rather than something a person recreates.

Create the vault and its configuration — permission model, role assignments, network rules, soft-delete retention, diagnostic settings — in Terraform, Bicep, or ARM, so every environment starts with identical guardrails. Then populate the secrets separately, because the values are the part infrastructure as code handles badly: Terraform writes secret values into its state file, and Bicep parameters need @secure() to stay out of deployment history.

For the secrets themselves, start from a vault whose structure is already right rather than from an empty one. Copying secrets between vaults covers the mechanics and the constraints, but the important choice for a new environment is what not to bring across: the names should come over, and the values should be replaced or generated for the environment they're going into. Cloning production into a development vault puts production credentials in the environment with the weakest controls, which is exactly backwards.

Short-lived environments have one extra trap. A per-branch or per-pull-request vault that gets deleted and recreated under the same name fails on creation, because the soft-deleted name is held for the retention period. Either include something unique in the name, or set the retention to 7 days at creation — and remember that the retention interval can't be changed afterwards.

What if every customer gets their own set of environments?

The arithmetic stops being friendly. Three environments for one application is three vaults; three environments for forty customers is 120, each holding dozens of secrets.

Microsoft's multitenancy guidance for Key Vault recommends a vault per tenant for SaaS solutions that need data isolation, and the security guidance repeats it. At that scale every operation in this article changes character. Adding a secret is no longer three writes, it's one per environment per customer. Rotating a shared vendor credential means finding every vault that holds it. Answering "does customer 37's staging environment have everything production has" stops being something anyone can do from the portal.

There's also a detail worth knowing before you script vault teardown at that scale: when a vault is soft-deleted, Azure RBAC role assignments and Event Grid subscriptions on it are deleted too, and recovering the vault doesn't bring them back. Recovery returns the secrets, not the access configuration.

Should you put all environments in one Key Vault?

No — and specifically not to make administration easier, which is the reason people usually reach for it.

The prefix approach (dev-Stripe--SecretKey and prod-Stripe--SecretKey in one vault) is the common shortcut, and it's the one Microsoft's documentation names and rejects. It also doesn't work on its own terms: the vault is where permissions, firewall rules, and diagnostics apply, so a prefix gives you a naming convention where you needed an access boundary. Anyone who can read one prefix can read all of them.

There are reasonable exceptions. An internal tool with a handful of secrets and one team doesn't need three vaults. But if the vault count is what's bothering you, that's a tooling problem, and it's cheaper to fix with tooling than by removing an isolation boundary you'll want back the first time a development credential leaks.

The better question is how to make several vaults easy to operate, which keeps the security architecture and the developer experience as separate problems.

What does a good multi-environment workflow look like?

Five capabilities cover nearly all of it.

• Search across vaults. Find a secret by name and see which vaults have it, without knowing in advance which vault to open.
• Write to a selected set of vaults. Add or update a secret across the vaults you choose, in one operation, with a different value per vault.
• Compare environments. See which vaults are missing a name the others have, before a deployment finds it.
• Start new environments from a known structure. Reproduce the secret names of an existing environment in a new vault, with values replaced or generated.
• Keep Azure's security model. Authenticate with Microsoft Entra ID and authorize with Key Vault RBAC, rather than moving secrets into a store with its own permissions to maintain.

KickCatch Secrets Manager provides these over the vaults you already have. It indexes secret names, tags, and structure so cross-vault search and comparison are possible, and it never retains secret values. Vaults are selected by naming pattern or by label, so an operation aimed at every production vault stays one action as the number of environments grows.

Frequently asked questions