Bahriya Tersane

Documentation / secrets-store-csi-driver

Secrets Store CSI driver

Installs the Kubernetes Secrets Store CSI driver, which mounts secrets from an external store into Pods as files. Applications then read a file, with no client for the store in their code.

The driver holds no secrets and speaks to no store. A provider for your store does that, and is installed separately. For OpenBao, the openbao chart can install its provider as a release of its own; it is the only provider this chart has been tested with. A provider must use the directory in providersDir, and it needs a host path too: run it in this namespace, as the only other workload there.

The chart provides:

  • the driver on every node, with its CSIDriver registration and its two CustomResourceDefinitions;
  • permissions no wider than the driver needs, with access to Kubernetes Secrets only if you ask for it;
  • the official images from the Mamluk registry, pinned by digest;
  • a NetworkPolicy and Prometheus metrics.

There can be only one driver in a cluster, and it is shared by every provider. The chart refuses to install a second.

Host access

The driver cannot meet the restricted Pod Security Standard, and no chart of this driver can. To place a volume into another Pod’s directory on the node it must run privileged. Install it only in a namespace of its own, and limit who can change that namespace, since anyone who can create Pods there can gain control of a node.

Container Privileged Host paths
secrets-store Yes, with Bidirectional mount propagation <kubelet>/pods, <kubelet>/plugins/csi-secrets-store/, the providers directory
node-driver-registrar No: every capability dropped, read-only root, running as root to write its socket <kubelet>/plugins/csi-secrets-store/, <kubelet>/plugins_registry/
liveness-probe No: every capability dropped, read-only root, running as root to reach the socket <kubelet>/plugins/csi-secrets-store/

<kubelet> is kubeletRootDir, /var/lib/kubelet by default. The providers directory is providersDir, /var/run/secrets-store-csi-providers by default. The driver uses no host network, PID or IPC namespace and no host port.

The Pods that mount secrets need none of this: they can run under restricted.

Admission policies

Any admission policy forbidding privileged containers or host paths will flag the driver, including the Compliance and Assurance Convoy’s no-privileged and no-host-path. That is by design: the driver cannot work otherwise.

The Convoy does not exempt this namespace by default, since the driver is optional, and on a cluster without it a namespace of that name would be open to anyone able to create it. If you run the Convoy and install the driver, exempt the namespace yourself. The Convoy then applies no policy at all in that namespace, not only those two, so keep it to this driver and limit who can create Pods in it.

Do this before installing the driver: a policy set to Deny would otherwise refuse the install. Set this in the values of the compliance chart. The list replaces the default as a whole, so it repeats the default, which may be longer in a later version of that chart: check it against helm show values, and add any namespaces of your own.

policies:
  exemptNamespaces:
    - kube-system
    - kube-node-lease
    - kube-public
    - tigera-operator
    - calico-system
    - calico-apiserver
    - longhorn-system
    - cert-manager
    - kong
    - kuma-system
    - logging
    - bahriya-system
    - kyverno
    - trivy-system
    - secrets-store-csi

Record the exemption, and the reason above, wherever your regime asks you to account for exceptions. An exempted namespace is not a passed control: the driver remains privileged.

Prerequisites

  • Kubernetes 1.33 or later.
  • A namespace for the driver alone, allowed the privileged Pod Security Standard.
  • A pull secret for 1x.ax named 1x-ax in that namespace, created from an image token as described in the access documentation.
  • A provider for your secrets store.

Installation

kubectl create namespace secrets-store-csi
kubectl label namespace secrets-store-csi pod-security.kubernetes.io/enforce=privileged

helm install secrets-store-csi-driver tersane/secrets-store-csi-driver \
  --namespace secrets-store-csi \
  --version <chart version>

kubectl --namespace secrets-store-csi rollout status daemonset/secrets-store-csi-driver

The chart refuses the default and kube-* namespaces.

Using it

Describe what to mount in a SecretProviderClass, in the application’s namespace, and mount it. This one is for OpenBao’s provider; each provider has its own parameters.

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: orders-db
  namespace: orders
spec:
  provider: openbao
  parameters:
    roleName: orders
    objects: |
      - objectName: password
        secretPath: kv/data/orders/db
        secretKey: password
---
apiVersion: v1
kind: Pod
metadata:
  name: orders
  namespace: orders
spec:
  serviceAccountName: orders
  containers:
    - name: orders
      image: registry.example.com/orders:1
      volumeMounts:
        - name: secrets
          mountPath: /secrets
          readOnly: true
  volumes:
    - name: secrets
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: orders-db

A Pod whose secrets cannot be read does not start: it stays in ContainerCreating, and kubectl describe pod gives the reason.

A Pod already running keeps its files while the driver restarts. A Pod starting on a node whose driver is down waits until the driver returns.

Rotation

Mounted files do not change by default. With enableSecretRotation, the driver reads each one again every rotationPollInterval and updates the file; your application must read the file again to see it. Every read goes to your secrets store, for every Pod using the driver, so choose the interval with your store’s capacity in mind.

Copying into Kubernetes Secrets

A SecretProviderClass can ask for a copy in a Kubernetes Secret with secretObjects, which works only with syncSecret.enabled.

Enabling it gives the driver permission to read and change every Secret in the cluster, and the copy is then an ordinary Secret, in etcd and readable by anyone who can read Secrets in that namespace. It gives up much of the reason to use this driver. Leave it off unless a workload can take a secret in no other way.

Which to use

You want Use
An application that can call OpenBao itself, with renewal and leases OpenBao’s API directly
A file, with no secret in Kubernetes This driver
A Kubernetes Secret, for software that can read nothing else External Secrets Operator, not syncSecret

Uninstalling

Uninstalling removes the driver and leaves the CustomResourceDefinitions, because deleting them would delete every SecretProviderClass in the cluster. Running Pods keep their files until they stop; new Pods using the driver do not start.

If the driver stops being maintained

The driver is a Kubernetes SIG project, separate from any provider. Applications depend on the SecretProviderClass and the mounted file only, so a replacement would change no application code, but moving to another mechanism means changing their volumes.

Values

Value Purpose Default
image, registrarImage, livenessProbeImage The images; the digest decides what runs From 1x.ax
imagePullSecrets The pull secret for 1x.ax, in the release’s namespace 1x-ax
kubeletRootDir The kubelet’s directory on the nodes /var/lib/kubelet
providersDir Where providers place their sockets on the nodes /var/run/secrets-store-csi-providers
enableSecretRotation / rotationPollInterval Updating mounted files false / 2m
syncSecret.enabled Copies into Kubernetes Secrets; read “Copying into Kubernetes Secrets” first false
tokenRequests Service account tokens passed to providers, by audience []
logVerbosity The driver’s log detail 0
resources.driver / .registrar / .livenessProbe Requests and limits See values.yaml
nodeSelector, tolerations, priorityClassName Scheduling; every taint is tolerated by default See values.yaml
metrics.enabled / .annotations Metrics on port 8095; prometheus.io annotations true / true
metrics.serviceMonitor.* A ServiceMonitor Off
networkPolicy.enabled / .metrics.from The NetworkPolicy; sources for the metrics port true / every source