Kyverno values#

Values under hw-kyverno configure the Kyverno cluster policies and the policy exceptions Hopsworks workloads need.

Generated from the Hopsworks Helm chart 5.0.14 (Hopsworks 5.0.13).

Deployed according to the first of these values that is set: global._hopsworks.kyverno.enabled, global._hopsworks.full_platform.

General#

Defaults as YAML

hw-kyverno:
  hopsworkslib: {}

hw-kyverno #: Type object, default {}. override hw-kyverno values
hw-kyverno.hopsworkslib #: Type object, default {}. override hopsworkslib values

policies#

Defaults as YAML

hw-kyverno:
  policies:
    addCertificatesVolume:
      autogenControllers: DaemonSet,Deployment,Job,StatefulSet
      cacertsConfigMap: ca-pemstore
      enabled: false
      envs: []
      extraAnnotations: []
      hopsworksProjectLabelKey: hopsworks.ai/project
      initContainers:
        extraAnnotations: []
        labels: []
      labels:
      - key: job-type
        value: check-image-exist
      - key: job-type
        value: tag
      - key: job-type
        value: list-tags
      - key: job-type
        value: docker-build
      - key: job-type
        value: delete
      - key: job-type
        value: git-command
      mountPath: ''
    prohibitHostPath:
      enabled: false

hw-kyverno.policies #

Type object. Configuration for Hopsworks Kyverno policies and exceptions

Default

addCertificatesVolume:
  autogenControllers: DaemonSet,Deployment,Job,StatefulSet
  cacertsConfigMap: ca-pemstore
  enabled: false
  envs: []
  extraAnnotations: []
  hopsworksProjectLabelKey: hopsworks.ai/project
  initContainers:
    extraAnnotations: []
    labels: []
  labels:
  - key: job-type
    value: check-image-exist
  - key: job-type
    value: tag
  - key: job-type
    value: list-tags
  - key: job-type
    value: docker-build
  - key: job-type
    value: delete
  - key: job-type
    value: git-command
  mountPath: ''
prohibitHostPath:
  enabled: false

hw-kyverno.policies.addCertificatesVolume #

Type object. Configuration to add custom certificates to pods as a mounted volume

Default

autogenControllers: DaemonSet,Deployment,Job,StatefulSet
cacertsConfigMap: ca-pemstore
enabled: false
envs: []
extraAnnotations: []
hopsworksProjectLabelKey: hopsworks.ai/project
initContainers:
  extraAnnotations: []
  labels: []
labels:
- key: job-type
  value: check-image-exist
- key: job-type
  value: tag
- key: job-type
  value: list-tags
- key: job-type
  value: docker-build
- key: job-type
  value: delete
- key: job-type
  value: git-command
mountPath: ''

hw-kyverno.policies.addCertificatesVolume.autogenControllers #

Type string, default "DaemonSet,Deployment,Job,StatefulSet". the list of controllers separated by comma to generate the policy for

hw-kyverno.policies.addCertificatesVolume.cacertsConfigMap #

Type string, default "ca-pemstore". The name of the configmap containing the certificates. The configmap should contains the ca-certificates.crt trusted by the user to replace the whole /etc/ssl/certs. It should also include the root certificate(s) if any defined for OAUTH or LDAP identity providers. Also, if using a hosted object storage with a custom CA, it should includes the java/cacerts to trust that object storage host and that require updating the hopsfs.namenode.jvmOpts = "-Djavax.net.ssl.trustStore=/etc/ssl/certs/my-cacerts -Djavax.net.ssl.trustStorePassword=changeit" and similarly for the hopsfs.datanode.jvmOpts.

hw-kyverno.policies.addCertificatesVolume.enabled #

Type bool, default false. Enable add certificates volume

hw-kyverno.policies.addCertificatesVolume.envs #

Type list, default []. The array of environment variables with their values that you would like to inject to the pods along side the certificates volume.

hw-kyverno.policies.addCertificatesVolume.extraAnnotations #

Type list, default []. The array of extra annotations to use when filtering which pods to inject the certificates volume into.

hw-kyverno.policies.addCertificatesVolume.hopsworksProjectLabelKey #

Type string, default "hopsworks.ai/project". the name of the label key to identify hopsworks user project namespaces

hw-kyverno.policies.addCertificatesVolume.initContainers #

Type object, default {"extraAnnotations":[],"labels":[]}. Opt-in for injecting the certificates volume into init containers. When enabled, a second mutate rule is rendered that targets spec.initContainers and is gated by an OR of the dedicated annotation (default kyverno-inject-certs-init=enabled), extraAnnotations, and labels configured below. Pods must opt in via at least one of these matchers.

hw-kyverno.policies.addCertificatesVolume.initContainers.extraAnnotations #

Type list, default []. Init-specific extra annotations that opt a pod's init containers into certificate volume injection. ORed with the dedicated init-container annotation and labels below. Defaults to empty so init injection is opt-in by design.

hw-kyverno.policies.addCertificatesVolume.initContainers.labels #

Type list, default []. Init-specific labels that opt a pod's init containers into certificate volume injection. ORed with the dedicated init-container annotation and extraAnnotations. Defaults to empty so third-party operator-injected init containers are not silently mutated.

hw-kyverno.policies.addCertificatesVolume.labels #

Type list. The array of labels to use when filtering which pods to inject the certificates volume into. The default list includes job-type=check-image-exist, job-type=tag, job-type=list-tags, job-type=docker-build, job-type=delete for docker operations, that is needed if using a registry with custom CA. The default list also includes job-type=git-command for git operations, that is needed if using a git host with custom CA.

Default

- key: job-type
  value: check-image-exist
- key: job-type
  value: tag
- key: job-type
  value: list-tags
- key: job-type
  value: docker-build
- key: job-type
  value: delete
- key: job-type
  value: git-command

hw-kyverno.policies.addCertificatesVolume.mountPath #

Type string, default "". Path to mount the certificates volume

hw-kyverno.policies.prohibitHostPath #

Type object, default {"enabled":false}. Configuration for disabling hostPath volumes in Pods

hw-kyverno.policies.prohibitHostPath.enabled #

Type bool, default false. Enable hostPath policy and relevant exceptions

policyExceptions#

Defaults as YAML

hw-kyverno:
  policyExceptions:
    airflow:
      enabled: true
    brewer:
      enabled: true
    dockerRegistryConfigurer:
      enabled: true
    filebeat:
      enabled: true
    jobs:
      enabled: true
    jupyter:
      enabled: true
    opensearch:
      enabled: true
    prometheusNodeExporter:
      enabled: true
    pythonDeployment:
      enabled: true
    pythonapp:
      enabled: true
    rondb:
      enabled: true
    spark:
      enabled: true
      rssAppName: rss-hops
    systemJobs:
      enabled: true
    terminal:
      enabled: true
    vllm:
      enabled: true

hw-kyverno.policyExceptions #

Type object. Configuration for PolicyExceptions to exempt specific services from Kyverno PSS Restricted policies

Default

airflow:
  enabled: true
brewer:
  enabled: true
dockerRegistryConfigurer:
  enabled: true
filebeat:
  enabled: true
jobs:
  enabled: true
jupyter:
  enabled: true
opensearch:
  enabled: true
prometheusNodeExporter:
  enabled: true
pythonDeployment:
  enabled: true
pythonapp:
  enabled: true
rondb:
  enabled: true
spark:
  enabled: true
  rssAppName: rss-hops
systemJobs:
  enabled: true
terminal:
  enabled: true
vllm:
  enabled: true

hw-kyverno.policyExceptions.airflow #

Type object, default {"enabled":true}. PolicyException for airflow-scheduler Deployment. Airflow has a mount-airflow-folders sidecar that requires privileged access for FUSE mounting of HopsFS. This exception is enabled by default and only activates when both the airflow service and hw-kyverno are enabled.

hw-kyverno.policyExceptions.airflow.enabled #

Type bool, default true. Enable PolicyException for airflow-scheduler. Set to false to disable even when airflow service is enabled.

hw-kyverno.policyExceptions.brewer #

Type object, default {"enabled":true}. PolicyException for brewer Deployment. Brewer has a hopsfsmount sidecar that requires privileged access for FUSE mounting of HopsFS. This exception is enabled by default and only activates when both the brewer service and hw-kyverno are enabled.

hw-kyverno.policyExceptions.brewer.enabled #

Type bool, default true. Enable PolicyException for brewer. Set to false to disable even when brewer service is enabled.

hw-kyverno.policyExceptions.dockerRegistryConfigurer #

Type object, default {"enabled":true}. PolicyException for docker-registry-configurer DaemonSet. This service requires privileged access, hostPID, hostNetwork, and hostPath to configure container runtimes on nodes.

hw-kyverno.policyExceptions.dockerRegistryConfigurer.enabled #

Type bool, default true. Enable PolicyException for docker-registry-configurer

hw-kyverno.policyExceptions.filebeat #

Type object, default {"enabled":true}. PolicyException for filebeat DaemonSet. Filebeat requires hostNetwork, hostPath volumes, and runs as root to collect logs from all nodes.

hw-kyverno.policyExceptions.filebeat.enabled #

Type bool, default true. Enable PolicyException for filebeat

hw-kyverno.policyExceptions.jobs #

Type object, default {"enabled":true}. PolicyException for user jobs. These pods contain a hopsfsmount sidecar that requires privileged access for FUSE mounting of HopsFS.

hw-kyverno.policyExceptions.jobs.enabled #

Type bool, default true. Enable PolicyException for user jobs

hw-kyverno.policyExceptions.jupyter #

Type object, default {"enabled":true}. PolicyException for Jupyter server deployments. These pods contain a hopsfsmount sidecar that requires privileged access for FUSE mounting of HopsFS.

hw-kyverno.policyExceptions.jupyter.enabled #

Type bool, default true. Enable PolicyException for Jupyter server deployments

hw-kyverno.policyExceptions.opensearch #

Type object, default {"enabled":true}. PolicyException for opensearch StatefulSet. OpenSearch requires a privileged init container to configure vm.max_map_count kernel parameter. Only needed when olk.opensearch.setVMMaxMapCount is true.

hw-kyverno.policyExceptions.opensearch.enabled #

Type bool, default true. Enable PolicyException for opensearch

hw-kyverno.policyExceptions.prometheusNodeExporter #

Type object, default {"enabled":true}. PolicyException for prometheus-node-exporter DaemonSet. Node exporter requires hostNetwork and hostPath to collect node-level metrics.

hw-kyverno.policyExceptions.prometheusNodeExporter.enabled #

Type bool, default true. Enable PolicyException for prometheus-node-exporter

hw-kyverno.policyExceptions.pythonDeployment #

Type object, default {"enabled":true}. PolicyException for Python (model-server: python) KServe serving deployments. Agent deployments inject a root, privileged hopsfsmount FUSE sidecar (HWORKS-2871) that does not meet the restricted policy requirements.

hw-kyverno.policyExceptions.pythonDeployment.enabled #

Type bool, default true. Enable PolicyException for Python model serving deployments

hw-kyverno.policyExceptions.pythonapp #

Type object, default {"enabled":true}. PolicyException for Python app deployments (custom apps, Streamlit, Gradio). These pods contain a hopsfsmount sidecar that requires privileged access for FUSE mounting of HopsFS.

hw-kyverno.policyExceptions.pythonapp.enabled #

Type bool, default true. Enable PolicyException for Python app deployments

hw-kyverno.policyExceptions.rondb #

Type object, default {"enabled":true}. PolicyException for RonDB mysqlds StatefulSet. Mysqlds uses the SYS_NICE capability for process scheduling priority tuning. Only needed when rondb.rondb.meta.mysqld.addSysNiceCapability is true.

hw-kyverno.policyExceptions.rondb.enabled #

Type bool, default true. Enable PolicyException for RonDB mysqlds

hw-kyverno.policyExceptions.spark #

Type object, default {"enabled":true,"rssAppName":"rss-hops"}. PolicyException for Spark-related resources including: (1) Spark driver and executor pods created by spark-operator - these pods have container-level security contexts but lack pod-level security context support in older spark-operator versions, and (2) RSS (Remote Shuffle Service) coordinator and shuffle server Deployments/StatefulSets - these are dynamically created by the Uniffle controller with security contexts configured via CRD spec.

hw-kyverno.policyExceptions.spark.enabled #

Type bool, default true. Enable PolicyException for Spark-related resources (spark-operator driver/executor pods and RSS coordinator/shuffle server Deployments/StatefulSets)

hw-kyverno.policyExceptions.spark.rssAppName #

Type string, default "rss-hops". The app name of the RemoteShuffleService resource

hw-kyverno.policyExceptions.systemJobs #

Type object, default {"enabled":true}. PolicyException for Hopsworks system jobs including docker operations (docker-build, check-image-exist, tag, delete, list-tags), image validation (check-image), and conda library operations (list-libraries, export-libraries, conda-search-libraries). These jobs require privileged access to run buildkit/podman for building container images.

hw-kyverno.policyExceptions.systemJobs.enabled #

Type bool, default true. Enable PolicyException for Hopsworks system jobs

hw-kyverno.policyExceptions.terminal #

Type object, default {"enabled":true}. PolicyException for terminal server deployments. These pods contain a hopsfsmount sidecar that requires privileged access for FUSE mounting of HopsFS.

hw-kyverno.policyExceptions.terminal.enabled #

Type bool, default true. Enable PolicyException for terminal server deployments

hw-kyverno.policyExceptions.vllm #

Type object, default {"enabled":true}. PolicyException for vLLM model serving deployments created by KServe. These pods are deployed with default KServe/vLLM configurations that may not meet all restricted policy requirements.

hw-kyverno.policyExceptions.vllm.enabled #

Type bool, default true. Enable PolicyException for vLLM model serving deployments