hsml.deployment #

Attach a tag to a deployment.

A tag consists of a pair. Tag names are unique identifiers across the whole cluster. The value of a tag can be any valid json - primitives, arrays or json objects.

Run the commit job of the feature view this deployment logs through.

For a view on the "job" transport this commits every chunk that reached HopsFS, including what a stopped or killed replica left in the staging directory; for a "realtime" view it runs the materialization job.

Create a feature monitoring config on the logging feature group of a feature view deployment.

Model deployments use create_model_monitoring() instead. Finish the returned builder with a detection window, a reference window or value, a comparison, and save().

Create a model monitoring config bound to this deployment's model.

Resolves the model's parent feature view from the registered provenance and delegates to feature_view.create_model_monitoring with this deployment's model_name and model_version already filled in. The resulting config targets the FV's logging feature group, filters by this model+version, and defaults the reference training dataset to the version that was used to train the model.

my_deployment = ms.get_deployment(name="my_deployment")
my_deployment.create_model_monitoring(
    name="psi_drift",
).with_detection_window(
    time_offset="1d", window_length="1d",
).with_reference_training_dataset(  # defaults to model's TD version
).compare_on_distribution(
    feature_name="amount", metric="PSI", threshold=0.2,
).save()

Delete the deployment.

Delete a tag attached to a deployment.

Print a JSON description of the deployment.

Download the artifact files served by the deployment.

Download the archived logs of this deployment from HopsFS.

Each running instance archives its own output to the project's Logs dataset without being asked: the container copies its log to Logs/Serving/<deployment_name>/ when it exits, is restarted, or is stopped, so a pod removed by scale-to-zero still leaves its output behind. Files are named <UTC yyyyMMdd-HHmmss>_<pod>_<component>.log, one per instance run.

local_paths = deployment.download_logs(latest=True)
for local_path in local_paths:
    print(open(local_path).read())

Get the base endpoint URL for this deployment.

Returns the base URL that can be used with external HTTP clients. This is the path-based routing base endpoint without any protocol-specific suffixes like :predict or /v1.

If Istio client is not available, returns None.

Examples:

deployment = ms.get_deployment("my_deployment")
url = deployment.get_endpoint_url()
# url = "https://host:port/v1/project/name"

Retrieve the feature view this deployment serves, or None.

A feature view deployment names its view in the deployment env vars; a model deployment resolves it through the model's provenance.

Get the KServe inference URL for standard model deployments.

Returns the full URL with :predict suffix for KServe inference protocol. This method only returns a URL for standard model deployments (non-vLLM, with a model attached).

If Istio client is not available, falls back to Hopsworks REST API path.

Examples:

deployment = ms.get_deployment("my_deployment")
url = deployment.get_inference_url()
# Use with any HTTP client
import requests
response = requests.post(url, json={"instances": [[1, 2, 3]]})

Prints the deployment logs of the predictor or transformer.

Only the live pods of a running deployment are read. Logs of a stopped deployment are whatever was saved to HopsFS beforehand, retrieved with meth:download_logs or the "Log history" section in the UI.

.. note:: Legacy: this method prints to stdout and returns None. New code (and any agent / scripted use) should call meth:read_logs for a string return value or meth:tail_logs for incremental streaming.

Retrieve the metadata object for the model being used by this deployment, or None when it has no model.

Get the feature monitoring configurations for the model deployed by this deployment.

For a model deployment these are the configs filtered by the model version; for a feature view deployment the configs on the feature view's logging feature group. Delegates to the underlying model's get_monitoring_configs method.

import hopsworks

project = hopsworks.login()

ms = project.get_model_serving()
my_deployment = ms.get_deployment("my_deployment")

fm_configs = my_deployment.get_monitoring_configs()

Get the OpenAI-compatible API URL for vLLM deployments.

Returns the URL for OpenAI-compatible API endpoints (e.g., /v1/chat/completions). This method only returns a URL for LLM (vLLM) deployments.

Examples:

deployment = ms.get_deployment("my_llm_deployment")
url = deployment.get_openai_url()
# url = "https://host:port/v1/project/name/v1"
# Then use: url + "/chat/completions"

Get the current state of the deployment.

Get the value of a tag attached to a deployment.

Retrieve all tags attached to a deployment.

Get url to the deployment in Hopsworks.

Get every configuration version this deployment has ever had, newest first.

Prepare this deployment for inference, so the first predict() does not.

Downloads the schema the deployment's revision names and connects the transport its API protocol selects: the gRPC channel, or for REST an open connection to the model's endpoint, made with a metadata GET. Calling it is optional: predict() prepares the schema and the channel the same way on its first call, through the same code, and opens its connection with the request itself. Call it when the first request should not pay for discovery or the connection setup, for instance when a serving process starts before it takes traffic.

No prediction is sent. A prediction can log rows and have application side effects, so it is not used as a warm-up.

Concurrent callers prepare once and share the result. A failure leaves the deployment unprepared and is raised to the caller, so it can be tried again.

# login into Hopsworks using hopsworks.login()
ms = project.get_model_serving()
my_deployment = ms.get_deployment("my_deployment")

my_deployment.init_predict()
predictions = my_deployment.predict(inputs=[[1, 2, 3, 4]])

Check whether the deployment is created.

Check whether the deployment is ready to handle inference requests.

Check whether the deployment is stopped.

Send inference requests to the deployment.

One of data or inputs parameters must be set. Setting both raises ModelServingException. When the deployment has a schema, the rows are encoded and validated against it before the request is sent, and a gRPC deployment sends them as one v2 tensor per field; the pod validates again regardless.

Examples:

# login into Hopsworks using hopsworks.login()

# get Hopsworks Model Serving handle
ms = project.get_model_serving()

# retrieve deployment by name
my_deployment = ms.get_deployment("my_deployment")

# (optional) retrieve model input example
my_model = project.get_model_registry()                                .get_model(my_deployment.model_name, my_deployment.model_version)

# make predictions using model inputs (single or batch)
predictions = my_deployment.predict(inputs=my_model.input_example)

# or using more sophisticated inference request payloads
data = { "instances": [ my_model.input_example ], "key2": "value2" }
predictions = my_deployment.predict(data)

Return deployment logs as a single plain-text string.

Programmatic counterpart to meth:get_logs. Suitable for agents and scripts: never prints, never short-circuits on deployment state. The default source="kubernetes" reads the live pods only; logs of a stopped deployment are whatever was saved beforehand and are archived to HopsFS and retrieved with meth:download_logs or the "Log history" section in the UI.

Re-infer the deployment schema from the current feature view and mark it pending.

Use after enabling logging or changing the view; save() then publishes the new schema as a new revision. Passed features are kept.

feature_view.enable_logging(extra_log_columns={"channel": "string"})

deployment = ms.get_deployment("fraud")
deployment.reinfer_schema()  # `channel` becomes an optional request field
deployment.save()  # publishes the schema as a new revision

Restart the deployment so it picks up the latest code and environment state.

If the deployment is already stopped, it is started in place.

Make an earlier version of this deployment the active one again.

Nothing is copied: the version's files are still where they were, and it keeps its number. This object is updated to the reactivated configuration. Rolling back to the version that is already active does nothing.

Rolling back while a new version is updating replaces that rollout with one for the target version; the instances that were serving keep serving until it is ready. An in-place edit of the active version cannot be undone this way: stop the deployment or wait for it to fail, then save the previous configuration.

Persist this deployment including the predictor and metadata to Model Serving.

On an existing deployment the active version is edited in place by default and keeps its number. With new_version the configuration is stored as a new version, numbered one above the highest the deployment ever had, and made active. A save that changes nothing does nothing in either mode: no version is created and the running instances are not restarted. Two cases always count as a change: a script read from outside the version (a HopsFS mount path or git) and a changed environment image. Use Deployment.restart to refresh the running instances on purpose.

deployment.predictor.scaling_configuration.min_instances = 2
deployment.predictor.scaling_configuration.max_instances = 4
deployment.save(new_version=True)
deployment.rollback(1)

Start the deployment.

Stop the deployment.

Yield only newly observed log chunks as plain text.

Client-side polling, not server-streaming: each tick calls meth:read_logs with a moving cursor and yields the portion not already seen. The Kubernetes path dedups per pod by overlapping the previous and current tail windows; the OpenSearch path (old backends) uses the timestamp + doc_id pair. Only the live pods are followed; the archived logs of a stopped deployment are retrieved with meth:download_logs.

for chunk in deployment.tail_logs(timeout=120):
    print(chunk, end="")