hsml.resources #
Resources #
Resource configuration for a predictor or transformer.
| PARAMETER | DESCRIPTION |
|---|---|
cores | Number of CPUs. TYPE: |
memory | Memory (MB) resources. TYPE: |
gpus | Number of GPUs. TYPE: |
disk | Local disk (GB) the component's container may use, unlimited when not set. Only valid on limits; the same amount is also requested. The model files are downloaded to the predictor's local disk, so a predictor's disk must fit them. On a cluster that gives vLLM deployments a volume for their model files, a vLLM predictor takes no disk. TYPE: |
cores property writable #
Number of CPUs to be allocated per instance.
disk property writable #
Local disk (GB) to be allocated per instance, unlimited when not set.
Only valid on limits. The model files are downloaded to the predictor's local disk, so a predictor's disk must fit them. On a cluster that gives vLLM deployments a volume for their model files, a vLLM predictor takes no disk.
gpus property writable #
Number of GPUs to be allocated per instance.
memory property writable #
Memory resources to be allocated per instance.
ComponentResources #
PredictorResources #
Bases: ComponentResources
TransformerResources #
Bases: ComponentResources