Introduction
In modern cloud-native applications running on Kubernetes, managing sensitive information such as database credentials, API keys, access tokens, and connection strings is a critical responsibility. Mishandling such data can expose systems to security vulnerabilities, unauthorized access, and compliance risks.
A common but risky practice is embedding secrets directly into application code or configuration files. This approach not only increases exposure but also makes rotation and management difficult.
Kubernetes provides a dedicated mechanism called Secrets, designed specifically to handle confidential data in a controlled and secure manner.
This guide walks through how to manage secrets securely in Kubernetes using a structured approach, practical configurations, and production-level best practices.
What are Kubernetes Secrets?
Kubernetes Secrets are objects used to store and manage sensitive data separately from application code.
They are commonly used for:
Unlike ConfigMaps, which store non-sensitive configuration, Secrets are intended specifically for confidential data.
Why Secure Secret Management Matters
Improper handling of secrets can lead to:
For example, exposing a database password in a public repository can allow attackers to directly access and manipulate your database.
Secure secret management ensures:
Step 1: Create a Kubernetes Secret
kubectl create secret generic db-secret \
--from-literal=username=admin \
--from-literal=password=MyStrongPassword
Explanation
kubectl create secret generic: Creates a generic secret object
db-secret: Name of the secret
--from-literal: Defines key-value pairs directly
This command stores the credentials inside the Kubernetes cluster.
Step 2: Inspect the Secret
kubectl get secrets
kubectl describe secret db-secret
Explanation
Note: Values are base64-encoded, not encrypted by default.
Step 3: Inject Secrets as Environment Variables
apiVersion: v1
kind: Pod
metadata:
name: app-pod
spec:
containers:
- name: app-container
image: nginx
env:
- name: DB_USERNAME
valueFrom:
secretKeyRef:
name: db-secret
key: username
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: db-secret
key: password
Explanation
env: Defines environment variables inside the container
secretKeyRef: References a key from a Kubernetes Secret
name: Secret name
key: Specific value inside the secret
This approach allows applications to access secrets without hardcoding them.
Step 4: Mount Secrets as Volumes
apiVersion: v1
kind: Pod
metadata:
name: app-pod-volume
spec:
containers:
- name: app-container
image: nginx
volumeMounts:
- name: secret-volume
mountPath: "/etc/secrets"
readOnly: true
volumes:
- name: secret-volume
secret:
secretName: db-secret
Explanation
volumeMounts: Mounts the secret inside the container
mountPath: Directory where secret files will appear
readOnly: Prevents modification
Each key becomes a file inside the directory
This is useful when applications expect secrets as files.
Step 5: Avoid Hardcoding Secrets in Code
// Avoid this approach
string password = "MyStrongPassword";
Explanation
Hardcoding secrets in application code:
Instead, always retrieve values from environment variables or mounted files.
Step 6: Enable Encryption at Rest
By default, Kubernetes stores secrets in etcd using base64 encoding, which is not secure encryption.
To secure secrets properly, enable encryption at rest using an encryption provider configuration.
Key Points
Configure EncryptionConfiguration in API server
Use providers like AES-CBC
Ensures secrets are encrypted in storage
This is essential for production environments.
Step 7: Control Access Using RBAC
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: secret-reader
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
Explanation
Role: Defines permissions
resources: secrets: Targets secret objects
verbs: get: Allows read-only access
RBAC ensures that only authorized users or services can access secrets.
Step 8: Use External Secret Management Systems
For advanced security, integrate Kubernetes with external secret managers such as:
Azure Key Vault
AWS Secrets Manager
HashiCorp Vault
Benefits
This approach is commonly used in enterprise environments.
Step 9: Implement Secret Rotation
Secrets should not remain static for long periods.
Best Practices
Rotate credentials periodically
Automate rotation where possible
Update dependent services without downtime
Regular rotation minimizes the impact of potential leaks.
Step 10: Real-World Scenario
Consider a microservices-based system:
Each service has its own database credentials
Secrets are stored in Kubernetes
Access is restricted using RBAC
External vault handles rotation
This setup improves isolation, security, and scalability.
Common Mistakes to Avoid
Storing secrets in source code repositories
Using base64 encoding as security
Granting broad access permissions
Skipping encryption at rest
Avoiding these mistakes strengthens your security posture.
Advantages of Kubernetes Secret Management
Centralized handling of sensitive data
Easy integration with containers
Supports multiple access patterns
Limitations to Consider
Base64 encoding is not encryption
Requires additional setup for full security
Needs proper access control configuration
When Should You Use Kubernetes Secrets?
Use Kubernetes Secrets when:
Managing confidential application data
Deploying applications in production
Enforcing secure configuration practices
Summary
Managing secrets securely in Kubernetes is a foundational aspect of building reliable and secure cloud-native applications. By leveraging Kubernetes Secrets, enabling encryption at rest, applying strict access controls through RBAC, and integrating with external secret management systems, organizations can significantly reduce security risks. A well-implemented secret management strategy not only protects sensitive data but also ensures scalability, compliance, and long-term maintainability of modern distributed systems.