Documentation / openbao

OpenBao

Installs OpenBao, secrets management with an API compatible with HashiCorp Vault’s: a cluster of nodes on integrated Raft storage, one active and the others standing by.

The chart provides:

one, three or five nodes, spread across the nodes of your cluster;
TLS on every connection, always, with a certificate authority of the release’s own made with cert-manager or a certificate of your own, and renewed certificates read without a restart;
unsealing by hand by default, so that no key is ever in the cluster, and optional automatic unsealing through a static key or the Transit engine of another OpenBao or Vault;
upstream’s plugins for an HSM through PKCS#11 and for AWS KMS, Azure Key Vault, Google Cloud KMS and OCI KMS, included for you to test: none has been run against a real HSM or cloud service, as “How the seals have been tested” sets out;
an audit log of every request on standard output, with values hashed;
snapshots to S3-compatible storage on a schedule, and restore into a new release;
a NetworkPolicy, Prometheus metrics, and optional access from outside the cluster through Kong with an allow-list;
optionally, the Agent Injector and OpenBao’s provider for the Secrets Store CSI driver;
the official image from the Mamluk registry, pinned by digest, and every Pod meeting the restricted Pod Security Standard, except the CSI provider, which needs a host path.

The chart never initialises the cluster and never holds a key share or a token. Policies, auth methods and secret engines are yours to configure, with the bao client or Terraform, once the cluster is running.

Prerequisites

Kubernetes 1.33 or later.
cert-manager, unless you provide the certificate in tls.existingSecret. It is included with the Bahriya Kubernetes Engine; on other Kubernetes, install it first.
A pull secret for 1x.ax named 1x-ax in the release’s namespace, created from an image token as described in the access documentation.
A StorageClass for the volumes: the cluster’s default, or one named in persistence.storageClass.
As many nodes of the Kubernetes cluster as OpenBao nodes: no two OpenBao nodes are placed on one node of the cluster.
Permission to create a ClusterRoleBinding, for system:auth-delegator, which OpenBao’s Kubernetes auth method needs. Set kubernetesAuth.clusterRoleBinding: false if you cannot, and create it separately.

Installation

Do not use --wait: no node is ready until you have initialised the cluster.

helm install vault tersane/openbao \
  --namespace vault --create-namespace \
  --version <chart version>

The release’s resources are named <release>-openbao (here vault-openbao), or after the release alone if its name already contains openbao.

Initialising

Initialise the cluster once. This prints the unseal key shares and the root token, once only. Keep them somewhere safe outside this cluster, with no one person holding enough shares to open it alone.

kubectl --namespace vault exec -it vault-openbao-0 -c openbao -- \
  bao operator init -key-shares=5 -key-threshold=3

Enter the threshold of shares into the first node, repeating the command with a different share each time. It reads the share without showing it.

kubectl --namespace vault exec -it vault-openbao-0 -c openbao -- bao operator unseal

The other nodes join by themselves. Enter the shares into each of them as well:

kubectl --namespace vault exec -it vault-openbao-1 -c openbao -- bao operator unseal
kubectl --namespace vault exec -it vault-openbao-2 -c openbao -- bao operator unseal

Every node is then ready. Check it, and revoke the root token once you have created the policies and credentials you need:

helm test vault --namespace vault

With an automatic seal, bao operator init prints recovery key shares instead, and each node opens itself.

Connecting

Clients in the release’s namespace reach https://vault-openbao:8200, any ready node, and a standby passes requests to the active node. vault-openbao-active reaches the active node alone. Clients trust the ca.crt in the Secret vault-openbao-tls.

Applications can read secrets in four ways:

You want	Use
An application that calls OpenBao itself, with renewal and leases	OpenBao’s API
A file, with no change to the application and no host access, while the application can run a second container	The Agent Injector
A file, with no second container and no secret in Kubernetes, accepting a privileged driver on every node	The CSI provider
A Kubernetes Secret, for software that can read nothing else	External Secrets Operator, a separate product this chart neither installs nor tests

Agent Injector

injector:
  enabled: true

The Injector adds an OpenBao Agent to Pods annotated for it, and the Agent writes their secrets to files. It uses HashiCorp’s vault-k8s (Mozilla Public License 2.0), which OpenBao’s own chart uses too: this is a component from HashiCorp, though the Agent itself is the OpenBao image the servers run.

The Injector may read webhook configurations, which it needs to load its certificate, and nothing else in the cluster.

It works only in namespaces you label, and no other Pod in the cluster ever reaches it. This release’s namespace, kube-system and cert-manager can never match, even if labelled, so that the Injector can always restart.

kubectl label namespace orders openbao.tersane/agent-injector=enabled

Admit each such namespace to the API in networkPolicy.from, and create a role for the Pod’s service account in OpenBao’s Kubernetes auth method. Then annotate the Pod:

metadata:
  annotations:
    vault.hashicorp.com/agent-inject: "true"
    vault.hashicorp.com/role: orders
    vault.hashicorp.com/agent-inject-secret-password: kv/data/orders/db
    vault.hashicorp.com/agent-inject-template-password: |
      {{- with secret "kv/data/orders/db" -}}{{ .Data.data.password }}{{- end -}}

The secret appears at /vault/secrets/password. The Agent containers meet the restricted Pod Security Standard. A Pod whose service account the role does not allow gets no secret, and stays in its init container.

While the Injector is down

With the default, injector.failurePolicy: Fail:

No Pod can be created in a labelled namespace, annotated or not. kubectl and controllers see failed calling webhook, and Deployments retry until it returns. Running Pods are not affected.
Pods in every other namespace are not affected.

With injector.failurePolicy: Ignore, Pods are created, but an annotated Pod then starts with no Agent and no secret, and nothing but its own failure tells you so. Choose Ignore only if your applications fail clearly without their secrets and you would rather they ran than waited.

Two replicas and a disruption budget keep one running during a node drain. They are placed on different nodes when they can be, so on a cluster of one node they protect nothing: a drain stops both, and a labelled namespace refuses Pods until the node returns. Each call to the webhook gives up after 5 seconds.

CSI provider

With the Secrets Store CSI driver, a Pod reads a secret as a mounted file, with no second container.

The provider is a release of its own, in the driver’s namespace, with the server off. It needs a host path, so admission policy must exempt the namespace it runs in, and exempting the namespace that holds your secrets store would turn off every policy around it. The chart refuses to enable it beside the server.

Install the secrets-store-csi-driver chart. Its documentation states the host access it needs and the exemption for the Compliance and Assurance Convoy, which applies to this provider too.

Put the server’s CA certificate in a Secret in that namespace:

kubectl --namespace vault get secret vault-openbao-tls -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt
kubectl --namespace secrets-store-csi create secret generic vault-ca --from-file=ca.crt=./ca.crt

Admit that namespace, and only that one, to the server’s API, in the server’s release. Without this, Pods using the provider do not start.

networkPolicy:
  from:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: secrets-store-csi

Install the provider:

helm install openbao-provider tersane/openbao --namespace secrets-store-csi \
  --set server.enabled=false --set csiProvider.enabled=true \
  --set csiProvider.address=https://vault-openbao.vault.svc:8200 \
  --set csiProvider.caSecret=vault-ca \
  --version <chart version>

Only the driver and this provider belong in that namespace. No admission policy applies there once it is exempt, so restrict who may create Pods in it, as the chart cannot.

The provider runs as root to write its socket to a directory on the node, with every capability dropped and a read-only root, and it is not privileged. Its host path is the driver’s providers directory alone. It may ask for a token of any service account in the cluster, which it needs to log in as the Pod being mounted, so anyone able to change it could do the same.

The provider comes from OpenBao’s project, is younger and less widely used than the server, and has had few releases. If it stopped being maintained, the API and the Agent Injector would remain, and moving means changing each application’s volumes. It is not a dependency of the server.

Sealing

Stored data is encrypted, and a node that starts holds no key until it is unsealed. Choose how with seal.type before you initialise the cluster.

`seal.type`	A node that starts	Where the key is
`shamir` (default)	Stays sealed until operators enter its key shares	With your operators, never in the cluster
`static`	Opens by itself	In a Secret in this namespace
`transit`	Opens by itself, while the other cluster answers	In another OpenBao or Vault
`pkcs11`	Opens by itself, while the HSM answers	In your HSM
`awskms`, `azurekeyvault`, `gcpckms`, `ocikms`	Opens by itself, while the service answers	In your cloud’s key management service

With shamir, every restart needs people. A node drain, a node reboot or an upgrade leaves each node it touches sealed until its shares are entered. While a majority stays unsealed the cluster keeps serving; if a majority restarts, it stops until the shares are entered. Weigh this against how you patch your cluster.

With static, anyone who can read the Secret and the volumes can open the data. OpenBao recommends it only where something else already protects the key. Consider who can read Secrets in this namespace, and whether Secrets are encrypted at rest.

Settings that are not secret go in seal.parameters, named as in OpenBao’s documentation of each seal. Each key of the Secret in seal.existingSecret is a further setting of the same name.

How the seals have been tested

Seal	Tested with
`shamir`, `static`	This chart, end to end
`transit`	This chart, end to end, with a second release of this chart as the provider
`pkcs11`	SoftHSM only, through the library path below. No hardware HSM has been tested.
`awskms`	A local stand-in for AWS KMS only. AWS itself has not been tested.
`azurekeyvault`, `gcpckms`, `ocikms`	Loading only: the plugin starts and tries to reach its service, and no more. None has initialised a cluster or been run against the real service. Treat them as untested.

The chart includes these plugins so that you can test them; their inclusion does not mean they have been proven with your HSM or cloud. Test the seal you choose in your own environment, including losing it, before you store anything you cannot lose. OpenBao’s documentation lists the HSMs its project has tested.

A static key

head -c 32 /dev/urandom | base64 > ./seal-key
kubectl --namespace vault create secret generic vault-seal \
  --from-file=current_key=./seal-key

seal:
  type: static
  existingSecret: vault-seal
  parameters:
    current_key_id: "2026-11"

Keep a copy of the key outside the cluster. Without it, neither the volumes nor the backups can be opened.

Transit

The other cluster needs the Transit engine, a key, and an orphan periodic token whose policy allows update on transit/encrypt/<key> and transit/decrypt/<key>. Mount its CA certificate if the release does not already trust it.

seal:
  type: transit
  existingSecret: vault-seal   # key `token`
  parameters:
    address: https://bao.example.com:8200
    key_name: unseal
    mount_path: transit/
    tls_ca_cert: /openbao/seal-ca/ca.crt
  extraVolumes:
    - name: seal-ca
      secret:
        secretName: seal-ca
  extraVolumeMounts:
    - name: seal-ca
      mountPath: /openbao/seal-ca

Allow this release’s namespace in the other cluster’s NetworkPolicy, if it has one.

An HSM

OpenBao’s PKCS#11 plugin comes with the chart; your vendor’s library does not. Provide it from an image of your own, through an init container copying it to a shared volume. OpenBao’s image is Alpine with gcompat, so a library built for glibc may need testing. The key must exist in the HSM before you initialise the cluster.

seal:
  type: pkcs11
  existingSecret: vault-hsm     # key `pin`
  parameters:
    lib: /openbao/hsm/libvendor.so
    token_label: OpenBao
    key_label: bao-root
  extraVolumes:
    - name: hsm
      emptyDir: {}
  extraVolumeMounts:
    - name: hsm
      mountPath: /openbao/hsm
  extraInitContainers:
    - name: hsm-library
      image: registry.example.com/hsm-library:1
      command: [cp, /lib/libvendor.so, /openbao/hsm/]
      volumeMounts:
        - name: hsm
          mountPath: /openbao/hsm
      securityContext:
        allowPrivilegeEscalation: false
        readOnlyRootFilesystem: true
        capabilities:
          drop: [ALL]

Pull the image with a credential named in imagePullSecrets.

A cloud KMS

seal:
  type: awskms
  parameters:
    region: me-central-1
    kms_key_id: alias/openbao

Give the nodes access with a workload identity, through serviceAccount.annotations, or with a key in seal.existingSecret (access_key and secret_key for AWS). seal.extraEnv and seal.extraVolumes carry anything else a service needs, such as the path of a credentials file.

Changing the seal later

Changing seal.type on an initialised cluster is a seal migration, which OpenBao performs with the old and new seals configured together and the key shares entered by hand. Follow OpenBao’s procedure and try it on a copy first: the chart does not automate it, and it has not been tested with this chart. Rotating a static key, with previous_key_id in seal.parameters and previous_key in the Secret, is described in OpenBao’s documentation of that seal.

Upgrades

The nodes update one at a time, from the highest number. Each waits until the one before is ready, so that a majority is never lost.

With shamir, enter the shares into each node as it returns, or the update stops there, which is safe. Do not use helm upgrade --wait, which cannot finish without the shares.

helm upgrade vault tersane/openbao --namespace vault --version <chart version>
kubectl --namespace vault get pods -w
kubectl --namespace vault exec -it vault-openbao-2 -c openbao -- bao operator unseal

The active node may restart before the others, and a standby takes over. Clients see an error for a few seconds, and should retry. Take a backup first.

Changing the node count

Keep an odd number of nodes: 1, 3 or 5. A running node is not restarted.

To add nodes, raise replicas. The new nodes join by themselves. With shamir, enter the shares into each.

To remove nodes, first remove each from Raft while it is still running. If one of them is active, hand over first with bao operator step-down. Then lower replicas and delete their volumes, or a node returning later would rejoin with old data. A majority must stay up throughout.

kubectl --namespace vault exec -it vault-openbao-0 -c openbao -- sh -c \
  'bao login && bao operator raft remove-peer vault-openbao-4 && bao operator raft remove-peer vault-openbao-3'
helm upgrade vault tersane/openbao --namespace vault --reuse-values --set replicas=3
kubectl --namespace vault delete pvc data-vault-openbao-3 data-vault-openbao-4

Storage

The chart fixes the volume size at install. To grow a volume, on a StorageClass that allows expansion, patch each claim and leave persistence.size as it is:

kubectl --namespace vault patch pvc data-vault-openbao-0 \
  -p '{"spec":{"resources":{"requests":{"storage":"20Gi"}}}}'

Uninstalling the release keeps the volumes.

Backups

A backup is a Raft snapshot. The data in it stays encrypted: restoring it needs the key shares, static key or seal of the cluster it came from, which are not in the backup. Keep those apart from the backups, and test a restore.

Create the access the backup logs in with, using a token that may do so:

bao auth enable kubernetes
bao write auth/kubernetes/config kubernetes_host=https://kubernetes.default.svc
bao policy write snapshot - <<'EOF'
path "sys/storage/raft/snapshot" { capabilities = ["read"] }
EOF
bao write auth/kubernetes/role/backup \
  bound_service_account_names=vault-openbao-backup \
  bound_service_account_namespaces=vault \
  policies=snapshot ttl=10m

The ServiceAccount is created with the backup. Store the S3 access key in a Secret with the keys access-key and secret-key, then:

backup:
  enabled: true
  bucket: my-bucket
  basePath: openbao/vault
  existingSecret: openbao-s3
  authRole: backup

Each backup is a directory named after its time, holding snapshot.snap and manifest.json, which is written last. A directory without a manifest is incomplete and is never restored. Run one now to check it:

kubectl --namespace vault create job --from=cronjob/vault-openbao-backup backup-now
kubectl --namespace vault logs -f job/backup-now

Restoring

A restore replaces everything in the release it runs in.

Install a new release, with the same seal as the original, and initialise it and unseal it as above. Its own shares and token are used only for this.

Put a token of the new release in a Secret: the root token it has just been given.

kubectl --namespace restored create secret generic restore-token \
  --from-file=token=./new-root-token

Enable the restore.

backup:
  bucket: my-bucket
  basePath: openbao/vault
  existingSecret: openbao-s3
restore:
  enabled: true
  backup: latest
  tokenSecret: restore-token

When the Job has finished, restart every node. Until then the node that was active stays ready and answers requests with decryption failed errors: it still holds the new release’s key, and the data no longer matches it. We observed this in testing; OpenBao’s documentation does not describe it.
```
kubectl --namespace restored delete pods \
  -l app.kubernetes.io/instance=restored,app.kubernetes.io/component=node
```
Unseal each node with the shares of the original cluster; an automatic seal with the original key opens them by itself. The new release’s shares no longer open it.
Log in with a token from the original cluster. Delete the Secret and the Job’s settings: upgrade with restore.enabled false.

The checksum in the manifest is checked first, and a backup that does not exist is refused with nothing changed.

External access

exposure:
  external:
    enabled: true
    host: vault.example.com
    tlsSecret: vault-example-com-tls
    ipAllowlist: [203.0.113.0/24]

Kong terminates HTTPS on the host name, connects to OpenBao over TLS, and admits only the listed addresses.

This version’s tests check what the chart renders for external access; they have not run it through a Kong ingress controller. Test it in your own environment before relying on it.

Network policy

Admits the API from Pods in the release’s namespace and from the sources in networkPolicy.from, and the Raft port between the nodes alone. The Kubernetes API, the seal’s service and object storage are reached by outgoing traffic, which it does not restrict.

Metrics

Prometheus metrics are at /v1/sys/metrics?format=prometheus on the API port, over TLS and without a token. A Prometheus outside the namespace needs a source in networkPolicy.from. metrics.serviceMonitor.enabled creates a ServiceMonitor.

Values

Value	Purpose	Default
`replicas`	The number of nodes: 1, 3 or 5	`3`
`image.repository` / `.tag`	The OpenBao image; the digest decides what runs	From 1x.ax
`toolsImage.repository` / `.tag`	The image of the backup, restore and test Pods, and of the plugins	From 1x.ax
`imagePullSecrets`	The pull secret for 1x.ax, in the release’s namespace	`1x-ax`
`seal.type`	How nodes obtain the key: see “Sealing”	`shamir`
`seal.parameters`	The seal’s settings that are not secret	`{}`
`seal.existingSecret`	A Secret whose keys are further settings of the seal	`""`
`seal.extraEnv` / `.extraVolumes` / `.extraVolumeMounts` / `.extraInitContainers`	What a seal needs besides its settings	Empty
`ui`	OpenBao’s web interface	`false`
`audit.enabled`	An audit device on standard output	`true`
`extraConfig`	Further server configuration, as HCL	`""`
`logLevel`	The server’s log level	`info`
`tls.existingSecret`	A certificate of your own (keys tls.crt, tls.key, ca.crt)	`""`
`tls.extraDnsNames`	Further names on the certificate	`[]`
`tls.reloadSeconds`	How often a renewed certificate is looked for	`300`
`kubernetesAuth.clusterRoleBinding`	Binds the nodes’ ServiceAccount to `system:auth-delegator`	`true`
`persistence.size` / `.storageClass`	Volume size per node, fixed at install; StorageClass	`10Gi` / the default class
`resources` / `jobResources`	The server’s; those of its other containers and Pods	See `values.yaml`
`podDisruptionBudget.enabled` / `.maxUnavailable`	Nodes a drain may take at once	`true` / `1`
`topologySpread.*`, `nodeSelector`, `tolerations`, `priorityClassName`	Scheduling	See `values.yaml`
`serviceAccount.annotations`	On the nodes’ ServiceAccount, such as a workload identity	`{}`
`backup.enabled`, `.bucket`, `.basePath`, `.endpoint`, `.region`, `.existingSecret`	Snapshots to S3-compatible storage	Off
`backup.authPath` / `.authRole`	The Kubernetes auth mount and role the backup logs in with	`kubernetes` / `""`
`backup.schedule` / `.timezone` / `.retentionDays`	When, and how long kept	`0 2 * * *` / `UTC` / `30`
`restore.enabled`, `.backup`, `.bucket`, `.basePath`, `.tokenSecret`	Loading a backup into this release	Off
`exposure.external.*`	The API on a host name through Kong	Off
`metrics.enabled` / `.annotations`	Metrics; prometheus.io annotations on the Pods	`true` / `true`
`metrics.serviceMonitor.*`	A ServiceMonitor	Off
`networkPolicy.enabled` / `.from`	The NetworkPolicy; further sources for the API	`true` / `[]`
`server.enabled`	The server; off only for a release of the CSI provider alone	`true`
`injector.enabled`, `.replicas`, `.failurePolicy`, `.authPath`	The Agent Injector	Off; `2`; `Fail`; `auth/kubernetes`
`injector.image`, `.resources`, `.agentResources`	Its image and resources, and each Agent’s	See `values.yaml`
`csiProvider.enabled`, `.address`, `.caSecret`, `.providersDir`, `.authPath`	The CSI provider, alone in a release	Off
`csiProvider.image`, `.resources`, `.tolerations`, `.nodeSelector`	Its image, resources and scheduling	See `values.yaml`