How to Manage Secrets in Kubernetes Securely

Introduction

In modern cloud-native applications running on Kubernetes, managing sensitive information such as database credentials, API keys, access tokens, and connection strings is a critical responsibility. Mishandling such data can expose systems to security vulnerabilities, unauthorized access, and compliance risks.

A common but risky practice is embedding secrets directly into application code or configuration files. This approach not only increases exposure but also makes rotation and management difficult.

Kubernetes provides a dedicated mechanism called Secrets, designed specifically to handle confidential data in a controlled and secure manner.

This guide walks through how to manage secrets securely in Kubernetes using a structured approach, practical configurations, and production-level best practices.

What are Kubernetes Secrets?

Kubernetes Secrets are objects used to store and manage sensitive data separately from application code.

They are commonly used for:

Database usernames and passwords
API tokens
SSH keys
TLS certificates

Unlike ConfigMaps, which store non-sensitive configuration, Secrets are intended specifically for confidential data.

Why Secure Secret Management Matters

Improper handling of secrets can lead to:

Data breaches
Unauthorized system access
Compromised applications

For example, exposing a database password in a public repository can allow attackers to directly access and manipulate your database.

Secure secret management ensures:

Controlled access
Better compliance with security standards
Reduced attack surface

Step 1: Create a Kubernetes Secret

kubectl create secret generic db-secret \
  --from-literal=username=admin \
  --from-literal=password=MyStrongPassword

Explanation

kubectl create secret generic: Creates a generic secret object
db-secret: Name of the secret
--from-literal: Defines key-value pairs directly

This command stores the credentials inside the Kubernetes cluster.

Step 2: Inspect the Secret

kubectl get secrets
kubectl describe secret db-secret

Explanation

get secrets: Lists all secrets in the namespace
describe secret: Displays detailed information

Note: Values are base64-encoded, not encrypted by default.

Step 3: Inject Secrets as Environment Variables

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app-container
    image: nginx
    env:
    - name: DB_USERNAME
      valueFrom:
        secretKeyRef:
          name: db-secret
          key: username
    - name: DB_PASSWORD
      valueFrom:
        secretKeyRef:
          name: db-secret
          key: password

Explanation

env: Defines environment variables inside the container
secretKeyRef: References a key from a Kubernetes Secret
name: Secret name
key: Specific value inside the secret

This approach allows applications to access secrets without hardcoding them.

Step 4: Mount Secrets as Volumes

apiVersion: v1
kind: Pod
metadata:
  name: app-pod-volume
spec:
  containers:
  - name: app-container
    image: nginx
    volumeMounts:
    - name: secret-volume
      mountPath: "/etc/secrets"
      readOnly: true
  volumes:
  - name: secret-volume
    secret:
      secretName: db-secret

Explanation

volumeMounts: Mounts the secret inside the container
mountPath: Directory where secret files will appear
readOnly: Prevents modification
Each key becomes a file inside the directory

This is useful when applications expect secrets as files.

Step 5: Avoid Hardcoding Secrets in Code

// Avoid this approach
string password = "MyStrongPassword";

Explanation

Hardcoding secrets in application code:

Exposes sensitive data
Makes rotation difficult
Increases risk if code is leaked

Instead, always retrieve values from environment variables or mounted files.

Step 6: Enable Encryption at Rest

By default, Kubernetes stores secrets in etcd using base64 encoding, which is not secure encryption.

To secure secrets properly, enable encryption at rest using an encryption provider configuration.

Key Points

Configure EncryptionConfiguration in API server
Use providers like AES-CBC
Ensures secrets are encrypted in storage

This is essential for production environments.

Step 7: Control Access Using RBAC

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: secret-reader
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]

Explanation

Role: Defines permissions
resources: secrets: Targets secret objects
verbs: get: Allows read-only access

RBAC ensures that only authorized users or services can access secrets.

Step 8: Use External Secret Management Systems

For advanced security, integrate Kubernetes with external secret managers such as:

Azure Key Vault
AWS Secrets Manager
HashiCorp Vault

Benefits

Centralized secret management
Automatic rotation
Strong encryption and auditing

This approach is commonly used in enterprise environments.

Step 9: Implement Secret Rotation

Secrets should not remain static for long periods.

Best Practices

Rotate credentials periodically
Automate rotation where possible
Update dependent services without downtime

Regular rotation minimizes the impact of potential leaks.

Step 10: Real-World Scenario

Consider a microservices-based system:

Each service has its own database credentials
Secrets are stored in Kubernetes
Access is restricted using RBAC
External vault handles rotation

This setup improves isolation, security, and scalability.

Common Mistakes to Avoid

Storing secrets in source code repositories
Using base64 encoding as security
Granting broad access permissions
Skipping encryption at rest

Avoiding these mistakes strengthens your security posture.

Advantages of Kubernetes Secret Management

Centralized handling of sensitive data
Easy integration with containers
Supports multiple access patterns

Limitations to Consider

Base64 encoding is not encryption
Requires additional setup for full security
Needs proper access control configuration

When Should You Use Kubernetes Secrets?

Use Kubernetes Secrets when:

Managing confidential application data
Deploying applications in production
Enforcing secure configuration practices

Summary

Managing secrets securely in Kubernetes is a foundational aspect of building reliable and secure cloud-native applications. By leveraging Kubernetes Secrets, enabling encryption at rest, applying strict access controls through RBAC, and integrating with external secret management systems, organizations can significantly reduce security risks. A well-implemented secret management strategy not only protects sensitive data but also ensures scalability, compliance, and long-term maintainability of modern distributed systems.