hsml.deployment #

Deployment #

NOT_FOUND_ERROR_CODE `class-attribute` `instance-attribute` #

NOT_FOUND_ERROR_CODE = 240000

Metadata object representing a deployment in Model Serving.

id `property` #

Id of the deployment.

name `property` `writable` #

Name of the deployment.

version `property` #

Version of the deployment.

description `property` `writable` #

Description of the deployment.

has_model `property` #

Whether the deployment has a model associated.

predictor `property` `writable` #

Predictor used in the deployment.

requested_instances `property` #

Total number of requested instances in the deployment.

model_name `property` `writable` #

Name of the model deployed by the predictor.

model_path `property` `writable` #

Model path deployed by the predictor.

model_version `property` `writable` #

Model version deployed by the predictor.

artifact_version `property` `writable` #

Artifact version deployed by the predictor.

Deprecated

Artifact versions are deprecated in favor of deployment versions.

artifact_files_path `property` #

Path of the artifact files deployed by the predictor.

artifact_path `property` #

Path of the model artifact deployed by the predictor.

Deprecated

Artifact versions are deprecated in favor of deployment versions.

model_server `property` `writable` #

Model server ran by the predictor.

serving_tool `property` `writable` #

Serving tool used to run the model server.

script_file `property` `writable` #

Script file used by the predictor.

config_file `property` `writable` #

Model server configuration file passed to the model deployment.

It can be accessed via CONFIG_FILE_PATH environment variable from a predictor or transformer script. For LLM deployments without a predictor script, this file is used to configure the vLLM engine.

resources `property` `writable` #

Resource configuration for the predictor.

inference_logger `property` `writable` #

Configuration of the inference logger attached to this predictor.

inference_batcher `property` `writable` #

Configuration of the inference batcher attached to this predictor.

transformer `property` `writable` #

Transformer configured in the predictor.

model_registry_id `property` `writable` #

Model Registry Id of the deployment.

created_at `property` #

Created at date of the predictor.

creator `property` #

Creator of the predictor.

api_protocol `property` `writable` #

API protocol enabled in the deployment (e.g., HTTP or GRPC).

environment `property` `writable` #

Name of inference environment.

project_namespace `property` `writable` #

Name of inference environment.

project_name `property` `writable` #

Name of the project the deployment belongs to.

scaling_configuration `property` `writable` #

Scaling configuration for the deployment.

save #

save(await_update: int | None = 600)

Persist this deployment including the predictor and metadata to Model Serving.

PARAMETER	DESCRIPTION
`await_update`	If the deployment is running, awaiting time (seconds) for the running instances to be updated. If the running instances are not updated within this timespan, the call to this method returns while the update in the background. TYPE: `int \| None` DEFAULT: `600`

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	In case the backend encounters an issue.

start #

start(await_running: int | None = 600)

Start the deployment.

PARAMETER	DESCRIPTION
`await_running`	Awaiting time (seconds) for the deployment to start. If the deployment has not started within this timespan, the call to this method returns while it deploys in the background. TYPE: `int \| None` DEFAULT: `600`

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	In case the backend encounters an issue.

stop #

stop(await_stopped: int | None = 600)

Stop the deployment.

PARAMETER	DESCRIPTION
`await_stopped`	Awaiting time (seconds) for the deployment to stop. If the deployment has not stopped within this timespan, the call to this method returns while it stopping in the background. TYPE: `int \| None` DEFAULT: `600`

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	In case the backend encounters an issue.

delete #

delete(force: bool = False)

Delete the deployment.

PARAMETER	DESCRIPTION
`force`	Force the deletion of the deployment. If the deployment is running, it will be stopped and deleted automatically. TYPE: `bool` DEFAULT: `False`

Warning

A call to this method does not ask for a second confirmation.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	In case the backend encounters an issue.

get_state #

get_state() -> PredictorState

Get the current state of the deployment.

RETURNS	DESCRIPTION
`PredictorState`	The state of the deployment.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	In case the backend encounters an issue.

is_created #

is_created() -> bool

Check whether the deployment is created.

RETURNS	DESCRIPTION
`bool`	Whether the deployment is created or not.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	In case the backend encounters an issue.

is_running #

is_running(
    or_idle: bool = True, or_updating: bool = True
) -> bool

Check whether the deployment is ready to handle inference requests.

PARAMETER	DESCRIPTION
`or_idle`	Whether the idle state is considered as running (default is True). TYPE: `bool` DEFAULT: `True`
`or_updating`	Whether the updating state is considered as running (default is True). TYPE: `bool` DEFAULT: `True`

RETURNS	DESCRIPTION
`bool`	Whether the deployment is ready or not.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	In case the backend encounters an issue.

is_stopped #

is_stopped(or_created: bool = True) -> bool

Check whether the deployment is stopped.

PARAMETER	DESCRIPTION
`or_created`	Whether the creating and created state is considered as stopped (default is True). TYPE: `bool` DEFAULT: `True`

RETURNS	DESCRIPTION
`bool`	Whether the deployment is stopped or not.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	In case the backend encounters an issue.

predict #

predict(
    data: dict | InferInput = None,
    inputs: list | dict = None,
) -> dict

Send inference requests to the deployment.

One of data or inputs parameters must be set. If both are set, inputs will be ignored.

PARAMETER	DESCRIPTION
`data`	Payload dictionary for the inference request including the model input(s). TYPE: `dict \| InferInput` DEFAULT: `None`
`inputs`	Model inputs used in the inference requests. TYPE: `list \| dict` DEFAULT: `None`

RETURNS	DESCRIPTION
`dict`	Inference response.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	In case the backend encounters an issue.

Examples:

# login into Hopsworks using hopsworks.login()

# get Hopsworks Model Serving handle
ms = project.get_model_serving()

# retrieve deployment by name
my_deployment = ms.get_deployment("my_deployment")

# (optional) retrieve model input example
my_model = project.get_model_registry()                                .get_model(my_deployment.model_name, my_deployment.model_version)

# make predictions using model inputs (single or batch)
predictions = my_deployment.predict(inputs=my_model.input_example)

# or using more sophisticated inference request payloads
data = { "instances": [ my_model.input_example ], "key2": "value2" }
predictions = my_deployment.predict(data)

get_model #

get_model()

Retrieve the metadata object for the model being used by this deployment.

download_artifact_files #

download_artifact_files(local_path: str | None = None)

Download the artifact files served by the deployment.

PARAMETER	DESCRIPTION
`local_path`	Path where to download the artifact files in the local filesystem. TYPE: `str \| None` DEFAULT: `None`

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	In case the backend encounters an issue.

get_logs #

get_logs(component: str = 'predictor', tail: int = 10)

Prints the deployment logs of the predictor or transformer.

PARAMETER	DESCRIPTION
`component`	Deployment component to get the logs from (e.g., predictor or transformer). TYPE: `str` DEFAULT: `'predictor'`
`tail`	Number of most recent lines to retrieve from the logs. TYPE: `int` DEFAULT: `10`

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	In case the backend encounters an issue.

get_url #

get_url()

Get url to the deployment in Hopsworks.

get_endpoint_url #

get_endpoint_url() -> str | None

Get the base endpoint URL for this deployment.

Returns the base URL that can be used with external HTTP clients. This is the path-based routing base endpoint without any protocol-specific suffixes like :predict or /v1.

If Istio client is not available, returns None.

RETURNS	DESCRIPTION
`str \| None`	Base endpoint URL, or `None` if unavailable.

Examples:

deployment = ms.get_deployment("my_deployment")
url = deployment.get_endpoint_url()
# url = "https://host:port/v1/project/name"

get_openai_url #

get_openai_url() -> str | None

Get the OpenAI-compatible API URL for vLLM deployments.

Returns the URL for OpenAI-compatible API endpoints (e.g., /v1/chat/completions). This method only returns a URL for LLM (vLLM) deployments.

RETURNS	DESCRIPTION
`str \| None`	OpenAI-compatible URL (base URL + "/v1"), or `None` if not a LLM deployment.

Examples:

deployment = ms.get_deployment("my_llm_deployment")
url = deployment.get_openai_url()
# url = "https://host:port/v1/project/name/v1"
# Then use: url + "/chat/completions"

get_inference_url #

get_inference_url() -> str | None

Get the KServe inference URL for standard model deployments.

Returns the full URL with :predict suffix for KServe inference protocol. This method only returns a URL for standard model deployments (non-vLLM, with a model attached).

If Istio client is not available, falls back to Hopsworks REST API path.

RETURNS	DESCRIPTION
`str \| None`	Inference URL with `:predict` suffix, or `None` if not a standard model deployment.

Examples:

deployment = ms.get_deployment("my_deployment")
url = deployment.get_inference_url()
# Use with any HTTP client
import requests
response = requests.post(url, json={"instances": [[1, 2, 3]]})

describe #

describe()

Print a JSON description of the deployment.

hsml.deployment #

Deployment #

NOT_FOUND_ERROR_CODE class-attribute instance-attribute #

id property #

name property writable #

version property #

description property writable #

has_model property #

predictor property writable #

requested_instances property #

model_name property writable #

model_path property writable #

model_version property writable #

artifact_version property writable #

artifact_files_path property #

artifact_path property #

model_server property writable #

serving_tool property writable #

script_file property writable #

config_file property writable #

resources property writable #

inference_logger property writable #

inference_batcher property writable #

transformer property writable #

model_registry_id property writable #

created_at property #

creator property #

api_protocol property writable #

environment property writable #

project_namespace property writable #

project_name property writable #

scaling_configuration property writable #

save #

start #

stop #

delete #

get_state #

is_created #

is_running #

is_stopped #

predict #

get_model #

download_artifact_files #

get_logs #

get_url #

get_endpoint_url #

get_openai_url #

get_inference_url #

describe #

NOT_FOUND_ERROR_CODE `class-attribute` `instance-attribute` #

id `property` #

name `property` `writable` #

version `property` #

description `property` `writable` #

has_model `property` #

predictor `property` `writable` #

requested_instances `property` #

model_name `property` `writable` #

model_path `property` `writable` #

model_version `property` `writable` #

artifact_version `property` `writable` #

artifact_files_path `property` #

artifact_path `property` #

model_server `property` `writable` #

serving_tool `property` `writable` #

script_file `property` `writable` #

config_file `property` `writable` #

resources `property` `writable` #

inference_logger `property` `writable` #

inference_batcher `property` `writable` #

transformer `property` `writable` #

model_registry_id `property` `writable` #

created_at `property` #

creator `property` #

api_protocol `property` `writable` #

environment `property` `writable` #

project_namespace `property` `writable` #

project_name `property` `writable` #

scaling_configuration `property` `writable` #