The short answer
Credentials belong in a managed secret store — AWS Secrets Manager, Azure Key Vault or HashiCorp Vault — retrieved at runtime by an identity rather than embedded in code or configuration. Scanning git history matters as much as prevention: a key committed years ago and since removed from the working tree is still in history and still valid unless rotated.
Key takeaways
- A secret removed from the working tree is still in git history
- Treat every leaked credential as compromised and rotate it
- Prefer short-lived identity-based access over stored secrets entirely
- Automated rotation is only useful if the failure path is tested
Part of our guide to Cloud security: the configuration is the attack surface.
Hard-coded credentials remain one of the most reliable ways into an organisation. They end up in repositories, CI configuration, container images and environment files, and they persist long after the person who added them has moved on.
The best secret is no secret
Before choosing a store, ask whether the secret needs to exist. Cloud platforms let workloads authenticate by identity rather than credential — an IAM role on AWS, a managed identity on Azure. The workload receives short-lived credentials automatically, and there is nothing to leak, rotate or store.
Use this wherever the target supports it. It removes the entire class of problem rather than managing it.
Secret stores are for what remains: third-party API keys, database passwords for engines without IAM authentication, certificates, and credentials for systems outside your cloud.
Choosing a store
AWS Secrets Manager — native rotation for supported database engines, IAM-based access, cross-region replication. Charged per secret, which matters at scale.
AWS Systems Manager Parameter Store — cheaper, suitable for configuration and lower-sensitivity values. No native rotation.
Azure Key Vault — secrets, keys and certificates together, integrated with Entra ID.
HashiCorp Vault — the most capable, particularly for dynamic short-lived credentials and multi-cloud estates, at the cost of running it yourself.
For most single-cloud organisations the native service is the right answer. Vault earns its operational cost when you need dynamic secrets or genuine multi-cloud consistency.
Retrieve at runtime
Fetch secrets when the application starts or when first needed, using the workload identity. Do not bake them into container images, and be careful with environment variables — they appear in process listings, crash dumps and some logging configurations.
Cache retrieved secrets in memory with a sensible TTL. Calling the secret store on every request adds latency and cost for no benefit.
Rotation, tested
Automated rotation is genuinely valuable and genuinely capable of causing outages. The failure mode is an application caching a secret indefinitely and continuing to use the old value after rotation.
Design for the overlap: support two valid credentials during a rotation window, have applications re-read on authentication failure, and test the rotation path in non-production before enabling it in production. Rotation that has never been exercised is a scheduled incident.
Scan history, not just the working tree
Two things are needed, and the second is the one that gets skipped.
Pre-commit and CI scanning stops new secrets being added.
History scanning finds what is already there. A key committed three years ago and deleted the following week is still in git history and, in most cases, still valid.
Scan the full history once, then treat every finding as compromised. Rotate rather than assess — determining whether a leaked credential was ever used is far more expensive than rotating it, and far less certain.
Removing a secret from git history is possible but disruptive to anyone with a clone. Rotation is almost always the correct response; history rewriting is only worth it for genuinely sensitive material.
