Skip to content

hsml.predictor #

Predictor #

Bases: DeployableComponent

Metadata object representing a predictor in Model Serving.

api_protocol property writable #

API protocol enabled in the predictor (e.g., HTTP or GRPC).

artifact_files_path property #

Path of the artifact files deployed by the predictor.

artifact_path property #

Path of the model artifact deployed by the predictor. Resolves to /Projects/{project_name}/Models/{name}/{version}/Artifacts/{artifact_version}/{name}{version}.zip.

artifact_version property writable #

Artifact version deployed by the predictor.

Deprecated

Artifact versions are deprecated in favor of deployment versions.

config_file property writable #

Model server configuration file passed to the model deployment.

It can be accessed via CONFIG_FILE_PATH environment variable from a predictor or transformer script. For LLM deployments without a predictor script, this file is used to configure the vLLM engine.

created_at property #

Created at date of the predictor.

creator property #

Creator of the predictor.

default_predictor property #

default_predictor: bool

Whether the library's default predictor serves this deployment.

description property writable #

Description of the predictor.

env_vars property writable #

Environment variables of the predictor.

environment property writable #

Name of the inference environment.

feature_view_name property #

feature_view_name: str | None

Feature view served by a feature view deployment, else None.

feature_view_version property #

feature_view_version: int | None

Feature view version served by a feature view deployment, else None.

git_auto_redeploy property writable #

git_auto_redeploy: bool

Whether the deployment is rolled to the branch HEAD when a new commit is pushed.

git_branch property writable #

Configured Git branch for this deployment.

git_current_commit property #

git_current_commit: str | None

Commit this deployment is currently running.

Read-only, server-managed: recorded when the deployment clones the repository and when auto-redeploy rolls it to a new commit.

git_provider property writable #

Configured Git provider for this deployment.

git_resolved_branch property #

git_resolved_branch: str | None

Branch the running clone resolved to.

Read-only. Only set when no branch was configured, in which case this is the repository's default branch that the deployment checked out.

git_url property writable #

Configured Git repository URL for this deployment.

has_feature_view property #

has_feature_view: bool

Whether this is a feature view deployment.

id property #

Id of the predictor.

inference_logger property writable #

Configuration of the inference logger attached to this predictor.

knative_mode property writable #

Whether this deployment runs in KServe Knative mode.

True selects Knative mode, which supports scale-to-zero and Knative-only autoscaling. False selects Standard mode, which requires at least one instance and autoscales on a CPU/memory metric between the minimum and maximum instances, or runs a fixed replica count when they are equal. None leaves the mode for the backend to decide: LLM (vLLM) deployments default to Standard, every other deployment defaults to Knative mode. Read the current mode from a saved deployment via this property. Setting it to None before a save call keeps the stored mode unchanged.

Adds Knative mode selection, ~=5.1.0

Deployments can now select between KServe Knative and Standard mode.

model_framework property writable #

Model framework of the model to be deployed by the predictor.

model_name property writable #

Name of the model deployed by the predictor.

model_path property writable #

Model path deployed by the predictor.

model_server property #

Model server used by the predictor.

model_version property writable #

Model version deployed by the predictor.

name property writable #

Name of the predictor.

project_name property writable #

Name of the project the deployment belongs to.

project_namespace property writable #

Kubernetes project namespace.

requested_instances property #

Total number of requested instances in the predictor.

schema property writable #

schema: DeploymentSchema | None

Deployment schema: the one set in this session, else the one the deployment's revision names, else None.

Reading a persisted schema downloads Deployments/<name>/resources/schema/<id>.json once per object. Setting a schema marks it pending; save() publishes it.

schema_id property #

schema_id: str | None

Id of the schema this deployment's revision serves, from its env vars.

script_file property writable #

Script file used to load and run the model.

serving_tool property writable #

Serving tool used to run the model server.

tracing property writable #

Tracing configuration attached to the predictor.

transformer property writable #

Transformer configuration attached to the predictor.

version property #

Version of the predictor.

vllm_image_tag property writable #

VLLM image tag override; None means use the cluster default.

vllm_variant property writable #

VLLM image variant for this predictor (VLLM or VLLM_OMNI).

deploy #

deploy() -> deployment.Deployment

Create a deployment for this predictor and persists it in the Model Serving.

RETURNS DESCRIPTION
deployment.Deployment

The deployment metadata object of a new or existing deployment.

Examples:

import hopsworks

project = hopsworks.login()

# get Hopsworks Model Registry handle
mr = project.get_model_registry()

# retrieve the trained model you want to deploy
my_model = mr.get_model("my_model", version=1)

# get Hopsworks Model Serving handle
ms = project.get_model_serving()

my_predictor = ms.create_predictor(my_model)
my_deployment = my_predictor.deploy()

print(my_deployment.get_state())

describe #

describe()

Print a JSON description of the predictor.

for_feature_view classmethod #

for_feature_view(
    feature_view: Any,
    name: str | None = None,
    training_dataset_version: int | None = None,
    passed_features: list[str] | None = None,
    schema: DeploymentSchema | dict | None = None,
    script_file: str | None = None,
    **kwargs: Any,
) -> Predictor

Build the predictor of a feature view deployment: the default predictor with no model.

Resolves the training dataset version from the last accessed training dataset when not given, checks that statistics-dependent transformations have one, infers the schema, and records the feature view identity in the predictor env vars. Further keyword arguments, such as resources, environment, or env_vars, pass through to Predictor.

PARAMETER DESCRIPTION
feature_view

The feature view to serve.

TYPE: Any

name

Deployment name; defaults to the view name and version without special characters.

TYPE: str | None DEFAULT: None

training_dataset_version

Training dataset whose statistics the transformations use.

TYPE: int | None DEFAULT: None

passed_features

Features of the view whose values clients send with each request.

TYPE: list[str] | None DEFAULT: None

schema

A refinement of the inferred deployment schema, keeping its fields.

TYPE: DeploymentSchema | dict | None DEFAULT: None

script_file

A script subclassing DefaultPredict that ends with the run_kserve_wrapper() hand-over.

TYPE: str | None DEFAULT: None

RETURNS DESCRIPTION
Predictor

The predictor, with the schema pending until the deployment is saved.

get_endpoint_url #

get_endpoint_url() -> str | None

Get the base endpoint URL for this predictor.

Returns the base URL that can be used with external HTTP clients. This is the path-based routing base endpoint without any protocol-specific suffixes like :predict or /v1.

If Istio client is not available, returns None (Hopsworks REST API doesn't support base-only endpoints).

RETURNS DESCRIPTION
str | None

Base endpoint URL, or None if unavailable.

Examples:

url = predictor.get_endpoint_url()
# url = "https://host:port/v1/project/name"

get_inference_url #

get_inference_url() -> str | None

Get the KServe inference URL for standard model deployments and feature view deployments.

Returns the full URL with :predict suffix for KServe inference protocol. This method only returns a URL for standard model deployments (non-vLLM, with a model attached) and for feature view deployments.

If Istio client is not available, falls back to Hopsworks REST API path for model deployments. A feature view deployment is only reachable through Istio, because the Hopsworks REST proxy forwards requests for deployments without a model to the pod root, not to the KServe predict route; None is returned when Istio is unavailable.

RETURNS DESCRIPTION
str | None

Inference URL with :predict suffix, or None if not a standard model deployment.

Examples:

url = predictor.get_inference_url()
# url = "https://host:port/v1/project/name/v1/models/name:predict"

get_openai_url #

get_openai_url() -> str | None

Get the OpenAI-compatible API URL for vLLM deployments.

Returns the URL for OpenAI-compatible API endpoints (e.g., /v1/chat/completions). This method only returns a URL for LLM (vLLM) deployments.

RETURNS DESCRIPTION
str | None

OpenAI-compatible URL (base URL + "/v1"), or None if not a LLM deployment.

Examples:

url = predictor.get_openai_compatible_url()
# url = "https://host:port/v1/project/name/v1"
# Then use: url + "/chat/completions"