Kueue values#

Values under hw-kueue configure Kueue, the job queueing controller, and the queues Hopsworks schedules jobs with. With kueue.enabled=false Hopsworks uses a Kueue installed outside its release; Kueue outside the Hopsworks namespace covers moving an existing cluster there.

Generated from the Hopsworks Helm chart 5.2.3 (Hopsworks 5.2.2).

Deployed according to the first of these values that is set: global._hopsworks.kueue.enabled, global._hopsworks.full_platform.

General#

Defaults as YAML

hw-kueue:
  enforcePrerequisiteCheck: true
  fullnameOverride: kueue
  hopsworkslib: {}
  imagePullPolicy: Always
  kueue:
    enabled: false
  kueueApiVersion: v1beta2
  resourceFlavours:
  - annotations: {}
    labels: {}
    name: default-flavor
    spec:
      nodeLabels:
        cloud.provider.com/region: europe
      nodeTaints: {}
      tolerations: {}
  topologies:
  - levels:
    - nodeLabel: cloud.provider.com/region
    - nodeLabel: cloud.provider.com/zone
    - nodeLabel: kubernetes.io/hostname
    name: default

hw-kueue #

Type object, default {}. override hw-kueue values

hw-kueue.enforcePrerequisiteCheck #

Type bool, default true. Refuse to render while the bundled Kueue 0.12 or its v1alpha1 storage is still on the cluster, or when no Kueue serving kueue.x-k8s.io/v1beta2 is installed. Set false for helm template.

hw-kueue.fullnameOverride #

Type string, default "kueue". Name prefix of the objects of release kueue, the Kueue Hopsworks uses (installed with fullnameOverride kueue): the Hopsworks bindings name its ClusterRoles.

hw-kueue.hopsworkslib #

Type object, default {}. override hopsworkslib values

hw-kueue.imagePullPolicy #

Type string, default "Always".

hw-kueue.kueue #

Type object, default {"enabled":false}. This chart ships no Kueue. kueue.enabled stays false, as the 5.1 chart's migrate-to-external.sh and files/upgrade-kueue.sh check it; true is refused.

hw-kueue.kueueApiVersion #

Type string, default "v1beta2". The kueue.x-k8s.io version the backend of this chart reads. Read by files/upgrade-kueue.sh to tell this chart from 5.2.1 and earlier; do not change.

hw-kueue.resourceFlavours #

Type list. List of ResourceFlavors

Default

- annotations: {}
  labels: {}
  name: default-flavor
  spec:
    nodeLabels:
      cloud.provider.com/region: europe
    nodeTaints: {}
    tolerations: {}

hw-kueue.topologies[0].levels[0].nodeLabel #

Type string, default "cloud.provider.com/region".

hw-kueue.topologies[0].levels[1].nodeLabel #

Type string, default "cloud.provider.com/zone".

hw-kueue.topologies[0].levels[2].nodeLabel #

Type string, default "kubernetes.io/hostname".

hw-kueue.topologies[0].name #

Type string, default "default".

clusterQueues#

Defaults as YAML

hw-kueue:
  clusterQueues:
  - annotations: {}
    labels: {}
    name: other
    spec:
      cohort: cluster
      fairSharing:
        weight: '1'
      namespaceSelector: {}
      queueingStrategy: BestEffortFIFO
      resourceGroups:
      - coveredResources:
        - cpu
        - memory
        - pods
        - nvidia.com/gpu
        flavors:
        - name: default-flavor
          resources:
          - name: cpu
            nominalQuota: 0
          - name: memory
            nominalQuota: 0
          - name: pods
            nominalQuota: 0
          - name: nvidia.com/gpu
            nominalQuota: 0

hw-kueue.clusterQueues[0].annotations #: Type object, default {}.
hw-kueue.clusterQueues[0].labels #: Type object, default {}.
hw-kueue.clusterQueues[0].name #: Type string, default "other".
hw-kueue.clusterQueues[0].spec.cohort #: Type string, default "cluster".
hw-kueue.clusterQueues[0].spec.fairSharing.weight #: Type string, default "1".
hw-kueue.clusterQueues[0].spec.namespaceSelector #: Type object, default {}.
hw-kueue.clusterQueues[0].spec.queueingStrategy #: Type string, default "BestEffortFIFO".
hw-kueue.clusterQueues[0].spec.resourceGroups[0].coveredResources[0] #: Type string, default "cpu".
hw-kueue.clusterQueues[0].spec.resourceGroups[0].coveredResources[1] #: Type string, default "memory".
hw-kueue.clusterQueues[0].spec.resourceGroups[0].coveredResources[2] #: Type string, default "pods".
hw-kueue.clusterQueues[0].spec.resourceGroups[0].coveredResources[3] #: Type string, default "nvidia.com/gpu".
hw-kueue.clusterQueues[0].spec.resourceGroups[0].flavors[0].name #: Type string, default "default-flavor".
hw-kueue.clusterQueues[0].spec.resourceGroups[0].flavors[0].resources[0].name #: Type string, default "cpu".
hw-kueue.clusterQueues[0].spec.resourceGroups[0].flavors[0].resources[0].nominalQuota #: Type int, default 0.
hw-kueue.clusterQueues[0].spec.resourceGroups[0].flavors[0].resources[1].name #: Type string, default "memory".
hw-kueue.clusterQueues[0].spec.resourceGroups[0].flavors[0].resources[1].nominalQuota #: Type int, default 0.
hw-kueue.clusterQueues[0].spec.resourceGroups[0].flavors[0].resources[2].name #: Type string, default "pods".
hw-kueue.clusterQueues[0].spec.resourceGroups[0].flavors[0].resources[2].nominalQuota #: Type int, default 0.
hw-kueue.clusterQueues[0].spec.resourceGroups[0].flavors[0].resources[3].name #: Type string, default "nvidia.com/gpu".
hw-kueue.clusterQueues[0].spec.resourceGroups[0].flavors[0].resources[3].nominalQuota #: Type int, default 0.

cohorts#

Defaults as YAML

hw-kueue:
  cohorts:
  - annotations: {}
    labels: {}
    name: cluster
    spec:
      resourceGroups:
      - coveredResources:
        - cpu
        - memory
        - pods
        - nvidia.com/gpu
        flavors:
        - name: default-flavor
          resources:
          - name: cpu
            nominalQuota: 100
          - name: memory
            nominalQuota: 200Gi
          - name: pods
            nominalQuota: 100
          - name: nvidia.com/gpu
            nominalQuota: 50

hw-kueue.cohorts[0].annotations #: Type object, default {}.
hw-kueue.cohorts[0].labels #: Type object, default {}.
hw-kueue.cohorts[0].name #: Type string, default "cluster".
hw-kueue.cohorts[0].spec.resourceGroups[0].coveredResources[0] #: Type string, default "cpu".
hw-kueue.cohorts[0].spec.resourceGroups[0].coveredResources[1] #: Type string, default "memory".
hw-kueue.cohorts[0].spec.resourceGroups[0].coveredResources[2] #: Type string, default "pods".
hw-kueue.cohorts[0].spec.resourceGroups[0].coveredResources[3] #: Type string, default "nvidia.com/gpu".
hw-kueue.cohorts[0].spec.resourceGroups[0].flavors[0].name #: Type string, default "default-flavor".
hw-kueue.cohorts[0].spec.resourceGroups[0].flavors[0].resources[0].name #: Type string, default "cpu".
hw-kueue.cohorts[0].spec.resourceGroups[0].flavors[0].resources[0].nominalQuota #: Type int, default 100.
hw-kueue.cohorts[0].spec.resourceGroups[0].flavors[0].resources[1].name #: Type string, default "memory".
hw-kueue.cohorts[0].spec.resourceGroups[0].flavors[0].resources[1].nominalQuota #: Type string, default "200Gi".
hw-kueue.cohorts[0].spec.resourceGroups[0].flavors[0].resources[2].name #: Type string, default "pods".
hw-kueue.cohorts[0].spec.resourceGroups[0].flavors[0].resources[2].nominalQuota #: Type int, default 100.
hw-kueue.cohorts[0].spec.resourceGroups[0].flavors[0].resources[3].name #: Type string, default "nvidia.com/gpu".
hw-kueue.cohorts[0].spec.resourceGroups[0].flavors[0].resources[3].nominalQuota #: Type int, default 50.

installJob#

Defaults as YAML

hw-kueue:
  installJob:
    configMapMountPath: /mnt/kueue-resources
    configMapName: kueue-objects-install-job-resources
    name: kueue-obj
    resources:
      limits:
        cpu: 100m
        memory: 250Mi
      requests:
        cpu: 100m
        memory: 250Mi
    ttlSecondsAfterFinished: null

hw-kueue.installJob.configMapMountPath #: Type string, default "/mnt/kueue-resources".
hw-kueue.installJob.configMapName #: Type string, default "kueue-objects-install-job-resources".
hw-kueue.installJob.name #: Type string, default "kueue-obj". install job name
hw-kueue.installJob.resources.limits.cpu #: Type string, default "100m".
hw-kueue.installJob.resources.limits.memory #: Type string, default "250Mi".
hw-kueue.installJob.resources.requests.cpu #: Type string, default "100m".
hw-kueue.installJob.resources.requests.memory #: Type string, default "250Mi".
hw-kueue.installJob.ttlSecondsAfterFinished #: Type string, default nil. TTL in seconds for the install-kueue-object Job. Overrides global default.

rbac#

Defaults as YAML

hw-kueue:
  rbac:
    clusterqueue:
      enabled: true
    cohort:
      enabled: true
    localqueue:
      enabled: true
    metricsReader:
      enabled: true
    resourceFlavors:
      enabled: true
    topologies:
      enabled: true
    workload:
      enabled: true

hw-kueue.rbac.clusterqueue.enabled #: Type bool, default true.
hw-kueue.rbac.cohort.enabled #: Type bool, default true.
hw-kueue.rbac.localqueue.enabled #: Type bool, default true.
hw-kueue.rbac.metricsReader.enabled #: Type bool, default true. Bind the Hopsworks Prometheus server's service account to Kueue's metrics-reader ClusterRole, so the chart's kueue scrape job is authorized
hw-kueue.rbac.resourceFlavors.enabled #: Type bool, default true.
hw-kueue.rbac.topologies.enabled #: Type bool, default true.
hw-kueue.rbac.workload.enabled #: Type bool, default true.