Secrets Management

Overview

Secrets Management is PAM for Applications (A-to-A). When App A talks to Database B, it needs a password. Historically, developers put this password in:

1. Source Code (GitHub)
2. Config Files (Text)
3. Environment Variables

All of these are insecure. Secrets Management Centralizes these credentials into a Vault, encrypts them, and rotates them automatically.

Architecture

Reference Pattern: The Secret Broker

Applications should never know the secret "at rest." They should request it "at runtime."

Key Decisions

Decision	Options	Recommendation	Notes / Gotchas
Central vs Native	Enterprise Vault (HashiCorp), Cloud Native (AWS Secrets)	Hybrid	Use AWS Secrets Manager for AWS apps. Use HashiCorp/CyberArk for On-Prem/Multi-Cloud.
Delivery	Sidecar, API, File Mount	Sidecar / File Mount	API requires code changes. Sidecars (k8s) inject secrets as files/env vars without code changes.
Rotation	Static, Dynamic	Dynamic	Dynamic Secrets generate a unique credential for every request and delete it after. Zero shared risk.
Dev Access	Read/Write, Write-Only	Write-Only	Developers should be able to put secrets in the vault, but not read Production secrets.
Detection	Git Scanning, Manual	Git Scanning	You must scan commits (pre-receive hook) to block secrets from entering repo history.

Implementation

Phase 1: Stop the Bleeding (Git Scanning)

• Implement a tool (TruffleHog, GitGuardian) to scan repositories.
• Block commits containing high-entropy strings (API keys).
• Goal: Stop new secrets from leaking.

Phase 2: Centralize (The Vault)

• Deploy a Secrets Manager.
• Move "Config File" passwords into the Vault.
• Update apps to fetch from Vault (or use a K8s operator).
• Goal: Encryption at rest.

Phase 3: Rotate & Dynamic

• Enable auto-rotation for Databases.
• Switch to Dynamic Secrets (e.g., Vault creates a temporary SQL user for each web request).
• Goal: Ephemeral credentials.

Risks

• The "Secret Zero" Problem: To get the secret from the Vault, the App needs a token (Secret Zero). How do you protect that? (Solution: Platform Identity like AWS IAM Roles or K8s Service Accounts).
• Performance: If the Vault is slow, the App is slow.
• Lock-in: Using AWS Secrets Manager creates deep coupling to AWS.
• DevOps Velocity: If adding a secret takes a Jira ticket and 3 days, devs will hardcode it. Make it self-service.

KPIs

• Hardcoded Secret Count: Number of secrets found in code repositories scanning (Target: 0).
• Rotation Coverage: Percentage of application secrets (DB, API) that are automatically rotated.
• Secret Age: Average age of secrets (Target: < 90 days).
• Vault Adoption: Percentage of applications retrieving secrets from the central vault vs. local config.