Bahriya Tersane

Documentation / keycloak

Keycloak

Installs Keycloak, open-source identity and access management: single sign-on for your applications with OpenID Connect and SAML, user federation with LDAP and Active Directory, identity brokering, and an administration console.

The chart installs codecentric’s keycloakx chart, unchanged, under the name engine, with settings for a production cluster on your own premises:

  • the image from the Mamluk registry, pinned by digest;
  • two replicas on different nodes, forming one cluster, with a PodDisruptionBudget so that a node can be drained without an outage;
  • every Pod meeting the restricted Pod Security Standard, with no Kubernetes API credential;
  • a NetworkPolicy that admits the ports Keycloak’s replicas use to form a cluster only from one another;
  • health checks and metrics on Keycloak’s management port, never through the ingress;
  • an upgrade to a new Keycloak version that stops every replica of the old one first, as Keycloak requires;
  • a required hostname, so that Keycloak never builds a link from the address a request happened to arrive on;
  • a documented way to add providers and themes of your own without building an image.

Prerequisites

  • Kubernetes 1.33 or later, with at least two schedulable nodes.
  • A database: MySQL 8.0 or 8.4, MariaDB, PostgreSQL, Microsoft SQL Server or Oracle, with an empty database and a user that owns it. The Tersane mysql chart provides one; see The database.
  • A namespace of its own, conventionally keycloak.
  • A pull secret for 1x.ax named 1x-ax in that namespace, created from an image token as described in the access documentation. To use a secret of another name, set engine.imagePullSecrets.
  • An ingress controller that terminates TLS for Keycloak’s hostname.

Installation

Create a Secret holding the database user’s password, and one holding a first administrator for the new database:

kubectl --namespace keycloak create secret generic keycloak-db \
  --from-literal=password='<database password>'
kubectl --namespace keycloak create secret generic keycloak-first-admin \
  --from-literal=username=admin --from-literal=password='<password>'

Write your settings in a values file:

engine:
  hostname: https://id.example.com
  bootstrapAdmin:
    existingSecret: keycloak-first-admin
  database:
    vendor: mysql
    hostname: keycloak-mysql.keycloak.svc.cluster.local
    port: 3306
    database: keycloak
    username: keycloak
    existingSecret: keycloak-db
  ingress:
    enabled: true
    rules:
      - host: id.example.com
        paths:
          - path: /
            pathType: Prefix
    tls:
      - hosts: [id.example.com]
        secretName: keycloak-tls

Then install:

helm install keycloak tersane/keycloak \
  --namespace keycloak --version <chart version> -f keycloak.yaml

Check that both replicas are ready, and run the chart’s test:

kubectl --namespace keycloak get statefulset keycloak
helm test keycloak --namespace keycloak

Sign in to https://id.example.com/admin/ with the first administrator, create a permanent administrator in the master realm, and delete the first one. Keycloak marks it as temporary.

The database

Keycloak keeps all of its state in the database: realms, users, clients, credentials and, since Keycloak 26, user sessions. Its replicas keep nothing of their own, which is why they are a StatefulSet without volumes, and why a session survives Keycloak being stopped entirely. Back up the database; there is nothing else to back up.

With the Tersane mysql chart, create the database and its user in the mysql release’s values:

mysql:
  databases:
    - name: keycloak
      user: keycloak
      existingSecret: keycloak-db

where keycloak-db is a Secret in the mysql release’s namespace holding the password under the key password, as Keycloak’s does. Point Keycloak at the release’s Service, as in the example above. Never run two Keycloak installations on one database.

Hostname and path

engine.hostname is the full URL at which users and applications reach Keycloak. Keycloak issues every token for this address and builds every link from it, whatever address a request arrived on. It is required.

Keycloak is served at the root of the hostname. Installations that began before Keycloak 17 often serve it under /auth; to keep that path, set both:

engine:
  hostname: https://id.example.com/auth
  http:
    relativePath: /auth

and use /auth/ as the Ingress path. Applications inside the cluster can reach Keycloak through its Service, http://<release>-keycloak-http.<namespace>.svc; the tokens it issues still name engine.hostname.

Ingress

Keycloak serves plain HTTP on port 8080, and the ingress controller terminates TLS. Keycloak reads the client’s address and protocol from the X-Forwarded-* headers that Kong and most ingress controllers send; for an ingress controller that sends the standard Forwarded header instead, set engine.proxy.mode: forwarded.

On a Bahriya Kubernetes Engine cluster the ingress class is kong, which is the default here. To restrict the administration console to some addresses, publish it on a second Ingress, engine.ingress.console, and attach a Kong ip-restriction plugin to that Ingress alone:

engine:
  ingress:
    console:
      enabled: true
      annotations:
        konghq.com/plugins: keycloak-console-ip-restriction
      rules:
        - host: id.example.com
          paths:
            - path: /admin
              pathType: Prefix
      tls:
        - hosts: [id.example.com]
          secretName: keycloak-tls

The health and metrics endpoints are on the management port, 9000, which no Ingress publishes.

Providers and themes

Keycloak loads providers (JAR files: authenticators, user storage, event listeners and themes) from /opt/keycloak/providers. The chart starts Keycloak with kc.sh start, which applies the providers it finds at each start, so a provider is added without building an image of Keycloak:

  1. Build an image of your own holding your JAR files and a cp command, for example on busybox, and push it to a registry your cluster pulls from.
  2. Copy the JAR files from it in an init container into a volume mounted at /opt/keycloak/providers:
engine:
  extraInitContainers: |
    - name: providers
      image: registry.example.com/keycloak-providers:v1
      command: ["sh", "-c", "cp /providers/*.jar /target/"]
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        allowPrivilegeEscalation: false
        capabilities:
          drop: ["ALL"]
      volumeMounts:
        - name: providers
          mountPath: /target
  extraVolumes: |
    - name: providers
      emptyDir: {}
  extraVolumeMounts: |
    - name: providers
      mountPath: /opt/keycloak/providers

Each provider image needs its own init container, all copying into the same volume. The security context is required in a namespace that enforces restricted.

Applying the providers takes Keycloak some seconds at each start. To avoid it, build an image of Keycloak with your providers and kc.sh build, set engine.image to it, and set engine.args: ["start", "--optimized"].

A provider that implements one of Keycloak’s internal service provider interfaces, which Keycloak’s log names as such at start, may stop working at any Keycloak upgrade. Test it against each new version before upgrading.

Environment variables

The chart sets Keycloak’s environment from engine.hostname, engine.logLevel and engine.bootstrapAdmin, through the keycloakx chart’s extraEnv. Add variables of your own with engine.env, a list in the form of a container’s env; do not set engine.extraEnv, which would replace the chart’s:

engine:
  env:
    - name: KC_HTTP_MAX_QUEUED_REQUESTS
      value: "1000"
    - name: MY_PROVIDER_API_KEY
      valueFrom:
        secretKeyRef:
          name: my-provider
          key: api-key

Variables for an entire Secret or ConfigMap can be added with engine.extraEnvFrom, a string as the keycloakx chart takes it.

Network policy

The chart’s NetworkPolicy admits:

  • port 8080, HTTP, from every source by default. To admit your ingress controller alone, name its namespace in networkPolicy.http.from. Applications that reach Keycloak inside the cluster then need adding too.
  • port 9000, health and metrics, from every source by default; networkPolicy.management.from narrows it.
  • ports 7800 and 57800, the cluster transport and its failure detection, from Keycloak’s own replicas only.

Outgoing connections are not restricted, because Keycloak may need to reach identity providers, mail servers and directories that only you know.

The keycloakx chart’s own NetworkPolicy does not admit the cluster ports, and replicas behind it cannot form a cluster. Leave engine.networkPolicy.enabled off.

Upgrades

Keycloak does not support replicas of two versions in one cluster, nor a replica of the earlier version on a database the new version has migrated. When an upgrade changes Keycloak’s image, a step before the upgrade therefore scales Keycloak to zero and waits for every replica to stop. The new version then starts and migrates the database. Keycloak is unavailable during that time: under a minute with a small database, and longer where the migration has more data to change. Sessions survive it.

An upgrade that leaves the image unchanged, such as a change of settings, replaces the replicas one at a time, without an outage.

The step compares the running image with the new one, reading the cluster at upgrade time with Helm’s lookup. Tools that render the chart without access to the cluster cannot run it. With such a tool, scale Keycloak to zero before changing its version:

kubectl --namespace keycloak scale statefulset keycloak --replicas=0

Back up the database before every upgrade that changes Keycloak’s version. Keycloak cannot be downgraded: after the new version has migrated the database, the way back is the backup.

Moving from the keycloakx chart

An installation of the keycloakx chart moves to this chart with its database:

  1. Install this chart as a new release, in the same namespace or another, with the same database settings and Keycloak version, and with engine.replicas: 0.
  2. Scale the existing release’s StatefulSet to zero. Keycloak is unavailable from this point.
  3. Upgrade this release with engine.replicas: 2, and move your Ingress to its Service.
  4. Once it has served for a while, uninstall the old release.

Settings to review on the way:

  • KC_HOSTNAME or --hostname becomes engine.hostname, and --hostname-strict=false is removed.
  • Variables Keycloak 26 ignores can be dropped: PROXY_ADDRESS_FORWARDING, KEYCLOAK_HTTP_PORT, KEYCLOAK_HTTPS_PORT, JGROUPS_DISCOVERY_PROTOCOL, JGROUPS_DISCOVERY_PROPERTIES, CACHE_OWNERS_COUNT, CACHE_OWNERS_AUTH_SESSIONS_COUNT and JDBC_PARAMS. Keycloak’s log level is KC_LOG_LEVEL, set by engine.logLevel; KC_LOGLEVEL is ignored.
  • Your other variables move from extraEnv to engine.env.
  • The path: keep /auth as described in Hostname and path if your applications use it.

Pod Security

Every Pod the chart creates meets the Kubernetes restricted Pod Security Standard: Keycloak’s replicas, the upgrade step and the helm test Pod. Each runs as a user other than root, cannot escalate its privileges, drops all Linux capabilities and runs under the container runtime’s default seccomp profile. Install it in a namespace that enforces restricted. Init containers you add must meet it too.

Parameters

Values under engine are passed to the keycloakx chart. Those below are set by this chart, or added by it; any other value of the keycloakx chart may be set as well, under engine, at your own discretion.

Keycloak

Parameter Description Default
engine.hostname The full URL at which Keycloak is reached. Required ""
engine.logLevel Keycloak’s log-level info
engine.bootstrapAdmin.existingSecret Secret with username and password for a first, temporary administrator ""
engine.env Environment variables of your own, as a container’s env []
engine.extraEnv Set by this chart from the values above. Do not set  
engine.args Keycloak’s arguments ["start"]
engine.replicas Replicas, each on its own node 2
engine.podDisruptionBudget Disruption budget maxUnavailable: 1
engine.resources Resources of each replica. The Java heap is 70% of the memory limit requests 500m/1536Mi, limits 2000m/2Gi
engine.http.relativePath The path Keycloak is served under /
engine.proxy.mode Proxy headers trusted: xforwarded or forwarded xforwarded
engine.nameOverride The name in Keycloak’s resource names keycloak

Database

Parameter Description Default
engine.database.vendor mysql, mariadb, postgres, mssql or oracle. Required not set
engine.database.hostname Database host not set
engine.database.port Database port not set
engine.database.database Database name not set
engine.database.username Database user not set
engine.database.existingSecret Secret holding the user’s password ""
engine.database.existingSecretKey The password’s key in that Secret password

Image

Parameter Description Default
engine.image.repository Image, with its tag 1x.ax/tersane/core/keycloak/keycloak, with the tag of the version the chart runs
engine.image.digest Digest, which decides what runs the digest of the version the chart runs
engine.image.tag The version label Set in each release
engine.imagePullSecrets Pull secrets [{name: 1x-ax}]

Security

Parameter Description Default
engine.podSecurityContext Pod security context restricted, group 1000
engine.securityContext Container security context restricted, user 1000
engine.serviceAccount.automountServiceAccountToken Mount an API credential false
engine.enableServiceLinks Service environment variables false
engine.networkPolicy.enabled The keycloakx chart’s NetworkPolicy. Leave off false
networkPolicy.enabled This chart’s NetworkPolicy true
networkPolicy.http.from Sources admitted to port 8080; empty admits all []
networkPolicy.management.from Sources admitted to port 9000; empty admits all []

Ingress and metrics

Parameter Description Default
engine.ingress.enabled Publish Keycloak false
engine.ingress.ingressClassName Ingress class kong
engine.ingress.console.enabled Publish the administration console on its own Ingress false
engine.service.httpsPort HTTPS port; none, as TLS ends at the ingress 0
engine.podAnnotations prometheus.io annotations for the metrics on port 9000 set

Upgrades

Parameter Description Default
recreateOnVersionChange.enabled Scale Keycloak to zero before an upgrade that changes its image true
recreateOnVersionChange.timeoutSeconds Seconds to wait for the replicas to stop 180
recreateOnVersionChange.image.repository The step’s image 1x.ax/tersane/core/kubernetes/kubectl
recreateOnVersionChange.resources The step’s resources requests 10m/32Mi, limits 100m/128Mi