Documentation / trivy-operator

Trivy Operator

Installs trivy-operator, which scans a Kubernetes cluster continuously and keeps the results as resources in the cluster:

• vulnerabilities in the images every workload runs, operating system packages and application libraries alike, Java libraries included;
• configuration audit of workloads, Services, Roles, NetworkPolicies, Ingresses and other resources against Kubernetes security checks;
• RBAC assessment of every Role and ClusterRole;
• node checks of each node’s Kubernetes configuration files and their permissions;
• compliance reports, against compliance specifications installed by a Convoy or written by you.

The chart installs trivy-operator’s own Helm chart, unchanged, under the name engine, with settings for a production cluster on your own premises:

• every image from the Mamluk registry, pinned by digest;
• one Trivy server holding the vulnerability database for every scan, so a scan downloads nothing;
• the Java database delivered as an image, which each node pulls once;
• the checks pinned by digest, so that the checks a cluster runs are fixed by the chart’s version;
• no call to any service other than the registry the images come from, and no scan that looks a package up online;
• every check recorded, passed as well as failed, so that a report can show a control met and not only one missed;
• at most three scans at a time, each with resource limits.

Prerequisites

• Kubernetes 1.36 or later. Scans mount the Java database as an image volume, which needs a container runtime that supports image volumes, such as containerd 2.1 or later.
• One trivy-operator per cluster. Its custom resources are cluster-wide; do not install a second release, in any namespace.
• A namespace of its own, conventionally trivy-system, that allows privileged Pods (see Pod Security below).
• A pull secret for 1x.ax named 1x-ax in that namespace, created from an image token as described in the access documentation. Every Pod the chart starts pulls its image with it, the Trivy server downloads the vulnerability database with it, and the operator downloads the checks with it.

Installation

helm install trivy-operator tersane/trivy-operator \
  --namespace trivy-system --create-namespace \
  --version <chart version>

Check that the operator and the Trivy server are ready:

kubectl --namespace trivy-system get deployments,statefulsets

The Trivy server is ready once it has downloaded the vulnerability database, about 140 MB. The first scans start then, and every workload in the cluster is scanned within minutes to hours, depending on how many there are. Read the results with:

kubectl get vulnerabilityreports --all-namespaces
kubectl get configauditreports --all-namespaces
kubectl get clusterrbacassessmentreports
kubectl get clusterinfraassessmentreports

How the checks and the databases reach the cluster

The checks are a bundle published by the Trivy project: the configuration, RBAC and node checks, and the commands with which the node checks read each node. The operator downloads it from 1x.ax/tersane/core/aquasecurity/trivy-checks when it starts, with the credentials of the pull secret 1x-ax. The chart pins it by digest, so a cluster runs the same checks until the chart is upgraded. Without it, no node checks run.

Scanning also needs two databases, both published by the Trivy project and kept current in the Mamluk registry:

• The vulnerability database (about 140 MB). The Trivy server downloads it from 1x.ax/tersane/core/aquasecurity/trivy-db when it starts and whenever a newer version is published, with the credentials of the pull secret 1x-ax. It is kept on the server’s own temporary volume and downloaded again after a restart.
• The Java database (about 1.5 GB), which identifies Java libraries packaged without their Maven metadata. It is published as the image 1x.ax/tersane/core/aquasecurity/trivy-java-db-volume:1, rebuilt daily under the same tag, and mounted read-only into every scan. Each node’s container runtime pulls it once and keeps it, fetching a new version only when one has been published; a scan never downloads it.

Clusters without access to 1x.ax

Copy the images, the checks bundle, the vulnerability database and the Java database image into a registry of your own, and point the chart at it. The checks bundle and the vulnerability database are OCI artefacts rather than images; tools such as skopeo copy them the same way. Copy the checks bundle with its digest preserved, at the tag engine.policiesBundle.tag names:

skopeo copy --all --preserve-digests \
  docker://1x.ax/tersane/core/aquasecurity/trivy-checks:<tag> \
  docker://registry.example.com/aquasecurity/trivy-checks:<tag>
skopeo copy --all --preserve-digests \
  docker://1x.ax/tersane/core/aquasecurity/trivy-db:2 \
  docker://registry.example.com/aquasecurity/trivy-db:2
skopeo copy --all \
  docker://1x.ax/tersane/core/aquasecurity/trivy-java-db-volume:1 \
  docker://registry.example.com/aquasecurity/trivy-java-db-volume:1

engine:
  policiesBundle:
    registry: registry.example.com/aquasecurity
  trivy:
    dbRegistry: registry.example.com/aquasecurity
  trivyOperator:
    scanJobCustomVolumes:
      - name: java-db
        image:
          reference: registry.example.com/aquasecurity/trivy-java-db-volume:1
          pullPolicy: Always

Repeat both copies regularly, daily for preference: the databases are only as current as the last copy. Then restart the Trivy server so that it reads the new vulnerability database at once:

kubectl --namespace trivy-system rollout restart statefulset trivy-server

The pull secret named in the chart must hold the credentials of your registry. If your registry’s TLS certificate is issued by a certificate authority of your own, the operator and the Trivy server must trust it; mount the authority’s certificate and point SSL_CERT_FILE at it, through engine.extraEnv, engine.volumes and engine.volumeMounts for the operator and engine.trivy.server.extraServerVolumes for the server. Each of these lists replaces the chart’s default as a whole, so repeat its entries.

Compliance specifications

The chart installs none of trivy-operator’s built-in compliance specifications. A Convoy installs the specifications of the regimes it evidences, and you may write your own as ClusterComplianceReport resources.

A compliance report summarises checks that have already run. A check that produced no result is shown in it as passed: a check whose name is misspelt, a control with no checks, and any check whose scan has not yet run. Read a compliance report as a view of the scan results, not as evidence that a control is met; a Convoy computes its results from the scan reports themselves.

The node checks have the same weakness. They read the kubelet’s settings from its command line, but a cluster built with kubeadm, the Bahriya Kubernetes Engine included, sets them in the kubelet’s configuration file. Those checks then collect nothing and report a pass whatever the setting is: notably AVD-KCV-0083 (protect kernel defaults), AVD-KCV-0092 (strong TLS cipher suites) and AVD-KCV-0068 (certificate file permissions). Read the kubelet’s effective configuration instead:

kubectl get --raw "/api/v1/nodes/<node>/proxy/configz"

AVD-KCV-0056 and AVD-KCV-0057 (container network interface file permissions and ownership) read the directories /etc/cni/net.d and /opt/cni/bin rather than the files in them, so a node with directories at the usual mode 755 fails AVD-KCV-0056 whatever the files’ own permissions.

Pod Security

The operator and the Trivy server meet the Kubernetes restricted Pod Security Standard: each runs as a user other than root, cannot escalate its privileges, drops all Linux capabilities and runs under the container runtime’s default seccomp profile.

The node checks do not: trivy-operator reads each node’s Kubernetes configuration with a short-lived Pod that runs as root, mounts the node’s /etc/kubernetes, /var/lib/kubelet, /var/lib/etcd and similar paths read-only, and shares the node’s process namespace. It drops every capability and cannot escalate its privileges, but host paths and the host’s process namespace are allowed only by the privileged standard. It runs in the release’s namespace, so that namespace must allow privileged Pods. The scans run there too.

To run without node checks, in a namespace that enforces restricted, set engine.operator.infraAssessmentScannerEnabled: false.

Uninstalling

helm uninstall trivy-operator --namespace trivy-system

Helm does not delete the custom resource definitions the chart installed, nor the reports already written. To remove them as well:

kubectl get crds -o name | grep aquasecurity.github.io | xargs kubectl delete

Deleting the definitions deletes every report and every compliance specification in the cluster.

Parameters

Values under engine are passed to trivy-operator’s chart. Those below are set by this chart; any other value of trivy-operator’s chart may be set as well, under engine, at your own discretion.

Images

Each image has a registry and a tag; the tag carries the image’s digest, which decides what runs. To pull from a mirror of your own, change each registry and keep the tags.

Parameter	Description	Default
engine.image	The operator	Set in each release
engine.trivy.image	The Trivy server and every scan	Set in each release
engine.nodeCollector	The node checks	1x.ax/tersane/core/aquasecurity/node-collector
engine.image.pullSecrets	Pull secrets of the operator	[{name: 1x-ax}]
engine.trivy.image.imagePullSecret	Pull secret of the server and the scans	1x-ax
engine.nodeCollector.imagePullSecret	Pull secret of the node checks	1x-ax

Checks

Parameter	Description	Default
engine.policiesBundle.registry	Registry of the checks bundle	1x.ax/tersane/core/aquasecurity
engine.policiesBundle.repository	Repository of the checks bundle	trivy-checks
engine.policiesBundle.tag	Tag and digest of the checks bundle	Set in each release
engine.extraEnv, engine.volumes, engine.volumeMounts	Mount the pull secret 1x-ax into the operator as its registry credentials	see values.yaml

Databases

Parameter	Description	Default
engine.trivy.dbRegistry	Registry of the vulnerability database	1x.ax/tersane/core/aquasecurity
engine.trivy.dbRepository	Repository of the vulnerability database	trivy-db
engine.trivy.server.extraServerVolumes	Mounts the pull secret 1x-ax into the server as its registry credentials	see values.yaml
engine.trivy.valuesFromConfigMap	The server’s environment, in a ConfigMap created by this chart: where its credentials are mounted, and the two settings of engine.trivy.configFile	trivy-server-env
engine.trivy.configFile	Trivy’s settings for every scan: no version check against check.trivy.dev, no usage data sent to Aqua Security	scan.skip-version-check: true, scan.disable-telemetry: true
engine.trivyOperator.scanJobCustomVolumes	The Java database image, mounted into every scan	1x.ax/tersane/core/aquasecurity/trivy-java-db-volume:1
engine.trivy.skipJavaDBUpdate	Scans never download the Java database	true
engine.trivy.offlineScan	Scans never look a package up online	true
engine.trivy.storageClassEnabled	The server keeps the database on a persistent volume	false

Scanning

Parameter	Description	Default
engine.operator.builtInTrivyServer	One Trivy server for every scan	true
engine.operator.scanJobsConcurrentLimit	Scans running at a time	3
engine.trivy.resources	Resources of each scan	requests 100m/100M, limits 500m/500M
engine.resources	Resources of the operator	requests 100m/256Mi, limits 1000m/1Gi
engine.podSecurityContext	Pod security context of the operator	non-root, RuntimeDefault seccomp
engine.trivy.server.podSecurityContext, engine.trivy.server.securityContext	Security contexts of the Trivy server	restricted
engine.trivyOperator.reportRecordFailedChecksOnly	Record failed checks only. A string: set "true" to record failures only	"false"
engine.operator.sbomGenerationEnabled	Keep a software bill of materials per image	false
engine.trivyOperator.useGCRServiceAccount	Credentials from Google Cloud’s metadata service	false
engine.operator.infraAssessmentScannerEnabled	Node checks	true
engine.compliance.specs	trivy-operator’s built-in compliance specifications to install	[]