hsml.deployment #

Attach a tag to a deployment.

A tag consists of a pair. Tag names are unique identifiers across the whole cluster. The value of a tag can be any valid json - primitives, arrays or json objects.

Create a feature monitoring config on the logging feature group of a feature view deployment.

Model deployments use create_model_monitoring() instead. Finish the returned builder with a detection window, a reference window or value, a comparison, and save().

Create a model monitoring config bound to this deployment's model.

Resolves the model's parent feature view from the registered provenance and delegates to feature_view.create_model_monitoring with this deployment's model_name and model_version already filled in. The resulting config targets the FV's logging feature group, filters by this model+version, and defaults the reference training dataset to the version that was used to train the model.

my_deployment = ms.get_deployment(name="my_deployment")
my_deployment.create_model_monitoring(
    name="psi_drift",
).with_detection_window(
    time_offset="1d", window_length="1d",
).with_reference_training_dataset(  # defaults to model's TD version
).compare_on_distribution(
    feature_name="amount", metric="PSI", threshold=0.2,
).save()

Delete the deployment.

Delete a tag attached to a deployment.

Print a JSON description of the deployment.

Download the artifact files served by the deployment.

Download the archived logs of this deployment from HopsFS.

Each running instance archives its own output to the project's Logs dataset without being asked: the container copies its log to Logs/Serving/<deployment_name>/ when it exits, is restarted, or is stopped, so a pod removed by scale-to-zero still leaves its output behind. Files are named <UTC yyyyMMdd-HHmmss>_<pod>_<component>.log, one per instance run.

local_paths = deployment.download_logs(latest=True)
for local_path in local_paths:
    print(open(local_path).read())

Get the base endpoint URL for this deployment.

Returns the base URL that can be used with external HTTP clients. This is the path-based routing base endpoint without any protocol-specific suffixes like :predict or /v1.

If Istio client is not available, returns None.

Examples:

deployment = ms.get_deployment("my_deployment")
url = deployment.get_endpoint_url()
# url = "https://host:port/v1/project/name"

Retrieve the feature view this deployment serves, or None.

A feature view deployment names its view in the deployment env vars; a model deployment resolves it through the model's provenance.

Get the KServe inference URL for standard model deployments.

Returns the full URL with :predict suffix for KServe inference protocol. This method only returns a URL for standard model deployments (non-vLLM, with a model attached).

If Istio client is not available, falls back to Hopsworks REST API path.

Examples:

deployment = ms.get_deployment("my_deployment")
url = deployment.get_inference_url()
# Use with any HTTP client
import requests
response = requests.post(url, json={"instances": [[1, 2, 3]]})

Prints the deployment logs of the predictor or transformer.

Only the live pods of a running deployment are read. Logs of a stopped deployment are whatever was saved to HopsFS beforehand, retrieved with meth:download_logs or the "Log history" section in the UI.

.. note:: Legacy: this method prints to stdout and returns None. New code (and any agent / scripted use) should call meth:read_logs for a string return value or meth:tail_logs for incremental streaming.

Retrieve the metadata object for the model being used by this deployment, or None when it has no model.

Get the feature monitoring configurations for the model deployed by this deployment.

For a model deployment these are the configs filtered by the model version; for a feature view deployment the configs on the feature view's logging feature group. Delegates to the underlying model's get_monitoring_configs method.

import hopsworks

project = hopsworks.login()

ms = project.get_model_serving()
my_deployment = ms.get_deployment("my_deployment")

fm_configs = my_deployment.get_monitoring_configs()

Get the OpenAI-compatible API URL for vLLM deployments.

Returns the URL for OpenAI-compatible API endpoints (e.g., /v1/chat/completions). This method only returns a URL for LLM (vLLM) deployments.

Examples:

deployment = ms.get_deployment("my_llm_deployment")
url = deployment.get_openai_url()
# url = "https://host:port/v1/project/name/v1"
# Then use: url + "/chat/completions"

Get the current state of the deployment.

Get the value of a tag attached to a deployment.

Retrieve all tags attached to a deployment.

Get url to the deployment in Hopsworks.

Check whether the deployment is created.

Check whether the deployment is ready to handle inference requests.

Check whether the deployment is stopped.

Send inference requests to the deployment.

One of data or inputs parameters must be set. Setting both raises ModelServingException. When the deployment has a schema and the protocol is REST, the rows are encoded and validated against it before the request is sent; the pod validates again regardless.

Examples:

# login into Hopsworks using hopsworks.login()

# get Hopsworks Model Serving handle
ms = project.get_model_serving()

# retrieve deployment by name
my_deployment = ms.get_deployment("my_deployment")

# (optional) retrieve model input example
my_model = project.get_model_registry()                                .get_model(my_deployment.model_name, my_deployment.model_version)

# make predictions using model inputs (single or batch)
predictions = my_deployment.predict(inputs=my_model.input_example)

# or using more sophisticated inference request payloads
data = { "instances": [ my_model.input_example ], "key2": "value2" }
predictions = my_deployment.predict(data)

Return deployment logs as a single plain-text string.

Programmatic counterpart to meth:get_logs. Suitable for agents and scripts: never prints, never short-circuits on deployment state. The default source="kubernetes" reads the live pods only; logs of a stopped deployment are whatever was saved beforehand and are archived to HopsFS and retrieved with meth:download_logs or the "Log history" section in the UI.

Re-infer the deployment schema from the current feature view and mark it pending.

Use after enabling logging or changing the view; save() then publishes the new schema as a new revision. Passed features are kept.

feature_view.enable_logging(extra_log_columns={"channel": "string"})

deployment = ms.get_deployment("fraud")
deployment.reinfer_schema()  # `channel` becomes an optional request field
deployment.save()  # publishes the schema as a new revision

Restart the deployment so it picks up the latest code and environment state.

If the deployment is already stopped, it is started in place.

Persist this deployment including the predictor and metadata to Model Serving.

Start the deployment.

Stop the deployment.

Yield only newly observed log chunks as plain text.

Client-side polling, not server-streaming: each tick calls meth:read_logs with a moving cursor and yields the portion not already seen. The Kubernetes path dedups per pod by overlapping the previous and current tail windows; the OpenSearch path (old backends) uses the timestamp + doc_id pair. Only the live pods are followed; the archived logs of a stopped deployment are retrieved with meth:download_logs.

for chunk in deployment.tail_logs(timeout=120):
    print(chunk, end="")