hsml.model_serving #

ModelServing #

project_id `property` #

Id of the project in which Model Serving is located.

project_name `property` #

Name of the project in which Model Serving is located.

project_path `property` #

Path of the project the registry is connected to.

create_deployment #

create_deployment(
    predictor: Predictor,
    name: str | None = None,
    environment: str | None = None,
    tags: tag.Tag
    | dict[str, Any]
    | list[tag.Tag | dict[str, Any]]
    | None = None,
) -> Deployment

Create a Deployment metadata object.

Example

# login into Hopsworks using hopsworks.login()

# get Hopsworks Model Registry handle
mr = project.get_model_registry()

# retrieve the trained model you want to deploy
my_model = mr.get_model("my_model", version=1)

# get Hopsworks Model Serving handle
ms = project.get_model_serving()

my_predictor = ms.create_predictor(my_model)

my_deployment = ms.create_deployment(my_predictor)
my_deployment.save()

Using the model object

# login into Hopsworks using hopsworks.login()

# get Hopsworks Model Registry handle
mr = project.get_model_registry()

# retrieve the trained model you want to deploy
my_model = mr.get_model("my_model", version=1)

my_deployment = my_model.deploy()

my_deployment.get_state().describe()

Using the Model Serving handle

# login into Hopsworks using hopsworks.login()

# get Hopsworks Model Registry handle
mr = project.get_model_registry()

# retrieve the trained model you want to deploy
my_model = mr.get_model("my_model", version=1)

# get Hopsworks Model Serving handle
ms = project.get_model_serving()

my_predictor = ms.create_predictor(my_model)

my_deployment = my_predictor.deploy()

my_deployment.get_state().describe()

Lazy

This method is lazy and does not persist any metadata or deploy any model. To create a deployment, call the save() method.

PARAMETER	DESCRIPTION
`predictor`	predictor to be used in the deployment TYPE: `Predictor`
`name`	name of the deployment TYPE: `str \| None` DEFAULT: `None`
`environment`	(Deprecated) The project Python environment to use. This argument will be ignored, use the argument `environment` in the `create_predictor()` or `create_endpoint()` methods instead. TYPE: `str \| None` DEFAULT: `None`
`tags`	Optionally the tags to attach to the deployment when it is created, in the same shapes accepted by feature groups. A single `Tag`, a `{"name": "owner", "value": "team-a"}` dict, or a list of either, for example `[{"name": "owner", "value": "team-a"}]`. The tags ride the create request, so any mandatory deployment tags missing from them cause the backend to reject the creation. TYPE: `tag.Tag \| dict[str, Any] \| list[tag.Tag \| dict[str, Any]] \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`Deployment`	The deployment metadata object.

create_endpoint #

create_endpoint(
    name: str,
    script_file: str,
    description: str | None = None,
    resources: PredictorResources | dict | None = None,
    inference_logger: InferenceLogger
    | dict
    | str
    | None = None,
    inference_batcher: InferenceBatcher
    | dict
    | None = None,
    api_protocol: str | None = IE.API_PROTOCOL_REST,
    environment: str | None = None,
    scaling_configuration: PredictorScalingConfig
    | dict
    | None = None,
    env_vars: dict | None = None,
    tracing: DeploymentTracingConfig | dict | None = None,
    git_url: str | None = None,
    git_provider: str | None = None,
    git_branch: str | None = None,
    git_auto_redeploy: bool = False,
    knative_mode: bool | None = None,
) -> Predictor

Create an Entrypoint metadata object.

Example

# login into Hopsworks using hopsworks.login()

# get Hopsworks Model Registry handle
ms = project.get_model_serving()

my_endpoint = ms.create_entrypoint(name="feature_server", entrypoint_file="feature_server.py")

my_deployment = my_endpoint.deploy()

Lazy

This method is lazy and does not persist any metadata or deploy any endpoint on its own. To create a deployment using this endpoint, call the deploy() method.

PARAMETER	DESCRIPTION
`name`	Name of the endpoint. TYPE: `str`
`script_file`	Path to a custom script file implementing a HTTP server, either local or already uploaded to HopsFS. TYPE: `str`
`description`	Description of the endpoint. TYPE: `str \| None` DEFAULT: `None`
`resources`	Resources to be allocated for the predictor. TYPE: `PredictorResources \| dict \| None` DEFAULT: `None`
`inference_logger`	Inference logger configuration. TYPE: `InferenceLogger \| dict \| str \| None` DEFAULT: `None`
`inference_batcher`	Inference batcher configuration. TYPE: `InferenceBatcher \| dict \| None` DEFAULT: `None`
`api_protocol`	API protocol to be enabled in the deployment (i.e., 'REST' or 'GRPC'). TYPE: `str \| None` DEFAULT: `IE.API_PROTOCOL_REST`
`environment`	The project Python environment to use TYPE: `str \| None` DEFAULT: `None`
`scaling_configuration`	Scaling configuration for the predictor. TYPE: `PredictorScalingConfig \| dict \| None` DEFAULT: `None`
`env_vars`	Environment variables to set on the predictor. TYPE: `dict \| None` DEFAULT: `None`
`tracing`	Tracing configuration for the endpoint. TYPE: `DeploymentTracingConfig \| dict \| None` DEFAULT: `None`
`git_url`	Optional Git repository URL for a git-backed endpoint. TYPE: `str \| None` DEFAULT: `None`
`git_provider`	Git provider for git-backed endpoints. TYPE: `str \| None` DEFAULT: `None`
`git_branch`	Optional branch to clone for git-backed endpoints. TYPE: `str \| None` DEFAULT: `None`
`git_auto_redeploy`	Roll the endpoint to the branch HEAD whenever a new commit is pushed. Only valid together with `git_url`. TYPE: `bool` DEFAULT: `False`
`knative_mode`	Whether to deploy in KServe Knative mode. `None` (default) lets the backend decide: every deployment defaults to Knative mode unless it is a vLLM deployment; on an update, `None` keeps the deployment's current mode. Standard mode does not scale to zero (minimum one instance). It autoscales on a CPU or memory metric between `min_instances` and `max_instances` (default: CPU at 80% up to the cluster maximum), and runs a fixed replica count without autoscaler when `min_instances == max_instances` (the default for LLM deployments). TYPE: `bool \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`Predictor`	The predictor metadata object.

create_feature_view_predictor #

create_feature_view_predictor(
    feature_view: Any,
    name: str | None = None,
    description: str | None = None,
    training_dataset_version: int | None = None,
    passed_features: list[str] | None = None,
    schema: DeploymentSchema | dict | None = None,
    script_file: str | None = None,
    resources: PredictorResources | dict | None = None,
    scaling_configuration: PredictorScalingConfig
    | dict
    | None = None,
    environment: str | None = None,
    env_vars: dict | None = None,
    tags: tag.Tag
    | dict[str, Any]
    | list[tag.Tag | dict[str, Any]]
    | None = None,
    knative_mode: bool | None = None,
) -> Predictor

Create the predictor of a feature view deployment, which serves transformed feature vectors without a model.

Same contract as a model deployment with the default predictor; the response carries one feature vector per request row under predictions and the column names under columns.

Lazy

This method is lazy and does not persist any metadata or deploy anything on its own. To create a deployment using this predictor, call the deploy() method.

PARAMETER	DESCRIPTION
`feature_view`	The feature view to serve. TYPE: `Any`
`name`	Deployment name; defaults to the view name and version without special characters. TYPE: `str \| None` DEFAULT: `None`
`description`	Deployment description. TYPE: `str \| None` DEFAULT: `None`
`training_dataset_version`	Training dataset whose statistics the transformations use; defaults to the last one accessed in this session. TYPE: `int \| None` DEFAULT: `None`
`passed_features`	Features of the view whose values clients send with each request. TYPE: `list[str] \| None` DEFAULT: `None`
`schema`	A refinement of the inferred deployment schema, keeping its fields. TYPE: `DeploymentSchema \| dict \| None` DEFAULT: `None`
`script_file`	A script subclassing `hsml.default_predictor.DefaultPredict` that ends with the `run_kserve_wrapper()` hand-over. TYPE: `str \| None` DEFAULT: `None`
`resources`	Resources to be allocated for the predictor. TYPE: `PredictorResources \| dict \| None` DEFAULT: `None`
`scaling_configuration`	Scaling configuration for the predictor. TYPE: `PredictorScalingConfig \| dict \| None` DEFAULT: `None`
`environment`	The inference environment to use. TYPE: `str \| None` DEFAULT: `None`
`env_vars`	Environment variables to set on the predictor. TYPE: `dict \| None` DEFAULT: `None`
`tags`	Tags to attach to the deployment when it is created. TYPE: `tag.Tag \| dict[str, Any] \| list[tag.Tag \| dict[str, Any]] \| None` DEFAULT: `None`
`knative_mode`	Whether to deploy in KServe Knative mode; see `ModelServing.create_predictor`. TYPE: `bool \| None` DEFAULT: `None`

Example

predictor = ms.create_feature_view_predictor(
    feature_view, name="transactionsfv", passed_features=["amount"]
)
deployment = ms.create_deployment(predictor)

RETURNS	DESCRIPTION
`Predictor`	The predictor metadata object.

create_predictor #

create_predictor(
    model: Model,
    name: str | None = None,
    artifact_version: str | None = None,
    serving_tool: str | None = None,
    script_file: str | None = None,
    config_file: str | None = None,
    resources: PredictorResources | dict | None = None,
    inference_logger: InferenceLogger
    | dict
    | str
    | None = None,
    inference_batcher: InferenceBatcher
    | dict
    | None = None,
    transformer: Transformer | dict | None = None,
    api_protocol: str | None = IE.API_PROTOCOL_REST,
    environment: str | None = None,
    scaling_configuration: PredictorScalingConfig
    | dict
    | None = None,
    env_vars: dict | None = None,
    vllm_variant: str | None = None,
    vllm_image_tag: str | None = None,
    tracing: DeploymentTracingConfig | dict | None = None,
    tags: tag.Tag
    | dict[str, Any]
    | list[tag.Tag | dict[str, Any]]
    | None = None,
    schema: DeploymentSchema | dict | None = None,
    passed_features: list[str] | None = None,
    default_predictor: bool | None = None,
    knative_mode: bool | None = None,
) -> Predictor

Create a Predictor metadata object.

Example

# login into Hopsworks using hopsworks.login()

# get Hopsworks Model Registry handle
mr = project.get_model_registry()

# retrieve the trained model you want to deploy
my_model = mr.get_model("my_model", version=1)

# get Hopsworks Model Serving handle
ms = project.get_model_serving()

my_predictor = ms.create_predictor(my_model)

my_deployment = my_predictor.deploy()

Lazy

This method is lazy and does not persist any metadata or deploy any model on its own. To create a deployment using this predictor, call the deploy() method.

PARAMETER	DESCRIPTION
`model`	Model to be deployed. TYPE: `Model`
`name`	Name of the predictor. TYPE: `str \| None` DEFAULT: `None`
`artifact_version`	(Deprecated) Version number of the model artifact to deploy, `CREATE` to create a new model artifact or `MODEL-ONLY` to reuse the shared artifact containing only the model files. TYPE: `str \| None` DEFAULT: `None`
`serving_tool`	Serving tool used to deploy the model server. TYPE: `str \| None` DEFAULT: `None`
`script_file`	Path to a custom predictor script implementing the Predict class, either local or already uploaded to HopsFS. The script must implement a `Predictor` class with a `predict(self, inputs)` method that takes the model inputs and returns the model predictions. The script is passed to the model deployment and can be accessed via the `SCRIPT_FILE_PATH` environment variable from within the predictor script. TYPE: `str \| None` DEFAULT: `None`
`config_file`	Model server configuration file to be passed to the model deployment, either local or already uploaded to HopsFS. It can be accessed via `CONFIG_FILE_PATH` environment variable from a predictor script. For LLM deployments without a predictor script, this file is used to configure the vLLM engine. TYPE: `str \| None` DEFAULT: `None`
`resources`	Resources to be allocated for the predictor. TYPE: `PredictorResources \| dict \| None` DEFAULT: `None`
`inference_logger`	Inference logger configuration. TYPE: `InferenceLogger \| dict \| str \| None` DEFAULT: `None`
`inference_batcher`	Inference batcher configuration. TYPE: `InferenceBatcher \| dict \| None` DEFAULT: `None`
`transformer`	Transformer to be deployed together with the predictor. TYPE: `Transformer \| dict \| None` DEFAULT: `None`
`api_protocol`	API protocol to be enabled in the deployment (i.e., 'REST' or 'GRPC'). TYPE: `str \| None` DEFAULT: `IE.API_PROTOCOL_REST`
`environment`	The project Python environment to use TYPE: `str \| None` DEFAULT: `None`
`scaling_configuration`	Scaling configuration for the predictor. TYPE: `PredictorScalingConfig \| dict \| None` DEFAULT: `None`
`env_vars`	Environment variables to set on the predictor. TYPE: `dict \| None` DEFAULT: `None`
`vllm_variant`	vLLM image variant for vLLM deployments. One of `'VLLM'` or `'VLLM_OMNI'`. Ignored for non-vLLM model servers. TYPE: `str \| None` DEFAULT: `None`
`vllm_image_tag`	vLLM image tag override. If set, it must be one of the tags `ModelServing.get_vllm_image_tags` returns for the matching `vllm_variant`; the two variants advertise different tags, and an unqualified call lists the standard vLLM ones. On a new deployment, `None` selects the newest advertised tag of that variant. On an update, `None` means "unchanged": the deployment keeps its current tag, so moving it to the cluster default takes naming that tag explicitly. A deployment also keeps its tag after an admin stops advertising it. Ignored for non-vLLM model servers. TYPE: `str \| None` DEFAULT: `None`
`tracing`	Tracing configuration for the predictor. TYPE: `DeploymentTracingConfig \| dict \| None` DEFAULT: `None`
`tags`	Optionally the tags to attach to the deployment when it is created, in the same shapes accepted by feature groups. A single `Tag`, a `{"name": "owner", "value": "team-a"}` dict, or a list of either, for example `[{"name": "owner", "value": "team-a"}]`. The tags ride the create request, so any mandatory deployment tags missing from them cause the backend to reject the creation. TYPE: `tag.Tag \| dict[str, Any] \| list[tag.Tag \| dict[str, Any]] \| None` DEFAULT: `None`
`schema`	Deployment schema describing the prediction requests; see `Model.deploy`. TYPE: `DeploymentSchema \| dict \| None` DEFAULT: `None`
`passed_features`	Feature view features whose values clients send with each request; see `Model.deploy`. TYPE: `list[str] \| None` DEFAULT: `None`
`default_predictor`	Whether the library's default predictor serves the model; see `Model.deploy`. TYPE: `bool \| None` DEFAULT: `None`
`knative_mode`	Whether to deploy in KServe Knative mode. `None` (default) lets the backend decide: LLM (vLLM) deployments default to Standard, every other deployment defaults to Knative mode; on an update, `None` keeps the deployment's current mode. Standard mode does not scale to zero (minimum one instance). It autoscales on a CPU or memory metric between `min_instances` and `max_instances` (default: CPU at 80% up to the cluster maximum), and runs a fixed replica count without autoscaler when `min_instances == max_instances` (the default for LLM deployments). TYPE: `bool \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`Predictor`	The predictor metadata object.

create_transformer #

create_transformer(
    script_file: str | None = None,
    resources: PredictorResources | dict | None = None,
    scaling_configuration: TransformerScalingConfig
    | dict
    | None = None,
    env_vars: dict | None = None,
) -> Transformer

Create a Transformer metadata object.

Example

# login into Hopsworks using hopsworks.login()

# get Hopsworks Model Serving handle
ms = project.get_model_serving()

# create my_transformer.py Python script
class Transformer(object):
    def __init__(self):
        ''' Initialization code goes here '''
        pass

    def preprocess(self, inputs):
        ''' Transform the requests inputs here. The object returned by this method will be used as model input to make predictions. '''
        return inputs

    def postprocess(self, outputs):
        ''' Transform the predictions computed by the model before returning a response '''
        return outputs

# Local path — auto-uploaded on save.
my_transformer = ms.create_transformer(script_file="my_transformer.py")

# Or an already-uploaded HopsFS path:
my_transformer = ms.create_transformer(
    script_file="/Projects/<project>/Resources/my_transformer.py"
)

# Or the Transformer class directly:
from hsml.transformer import Transformer
my_transformer = Transformer(script_file="my_transformer.py")

Create a deployment with the transformer

my_predictor = ms.create_predictor(transformer=my_transformer)
my_deployment = my_predictor.deploy()

# or
my_deployment = ms.create_deployment(my_predictor, transformer=my_transformer)
my_deployment.save()

Lazy

This method is lazy and does not persist any metadata or deploy any transformer. To create a deployment using this transformer, set it in the predictor.transformer property.

PARAMETER	DESCRIPTION
`script_file`	Path to a custom predictor script implementing the Transformer class, either local or already uploaded to HopsFS. TYPE: `str \| None` DEFAULT: `None`
`resources`	Resources to be allocated for the transformer. TYPE: `PredictorResources \| dict \| None` DEFAULT: `None`
`scaling_configuration`	Scaling configuration for the transformer. TYPE: `TransformerScalingConfig \| dict \| None` DEFAULT: `None`
`env_vars`	Environment variables to set on the transformer. TYPE: `dict \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`Transformer`	The transformer metadata object.

deploy_agent #

deploy_agent(
    entry: str,
    name: str | None = None,
    requirements: str | None = None,
    environment: str | None = None,
    upload_dir: str = "Resources/agents",
    description: str | None = None,
    resources: PredictorResources | dict | None = None,
    inference_logger: InferenceLogger
    | dict
    | str
    | None = None,
    inference_batcher: InferenceBatcher
    | dict
    | None = None,
    api_protocol: str | None = IE.API_PROTOCOL_REST,
    scaling_configuration: PredictorScalingConfig
    | dict
    | None = None,
    tracing: DeploymentTracingConfig | dict | None = None,
    git_url: str | None = None,
    git_provider: str | None = None,
    git_branch: str | None = None,
    git_auto_redeploy: bool = False,
    knative_mode: bool | None = None,
) -> Deployment

Deploy a Python script or package as an agent.

The agent is created on first call and updated on subsequent calls. Each call uploads the latest local code, refreshes the Python environment, and rewrites the deployment's predictor metadata to reflect the arguments passed in — including any unspecified arguments, which fall back to their defaults. The deployment's running state is left untouched; call start() after the first deploy and restart() to roll a running agent onto the new code. Works the same whether invoked from outside or inside a Hopsworks cluster.

Pass either a .py script or a directory containing a pyproject.toml. For a script, the file is uploaded and run directly. For a package, a wheel is built locally with the project's PEP 517 backend, uploaded, and installed; a small runner module invokes the package via runpy.run_module. When git_url is set, entry must be a safe relative .py path inside the repository and is used as-is.

ms = project.get_model_serving()

agent = ms.deploy_agent(entry="my_agent.py")
agent.start() # or agent.restart()

# iterate: edit code locally, push, then roll the running agent onto it
agent = ms.deploy_agent(entry="my_agent.py")
agent.restart()

PARAMETER	DESCRIPTION
`entry`	Local path to a `.py` script or to a directory containing `pyproject.toml`. TYPE: `str`
`name`	Name of the deployment, also used as the default Python environment name. Defaults to the basename of `entry` (without the `.py` extension for scripts). Must match `[A-Za-z0-9_-]+`. TYPE: `str \| None` DEFAULT: `None`
`requirements`	Local path to a `requirements.txt` to install into the environment. TYPE: `str \| None` DEFAULT: `None`
`environment`	Name of the Python environment to use; defaults to `name`. Created if it does not exist. Must match `[A-Za-z0-9_-]+`. TYPE: `str \| None` DEFAULT: `None`
`upload_dir`	Directory in the Hopsworks Filesystem under which agent files are placed; the agent gets its own subdirectory `<upload_dir>/<name>`. TYPE: `str` DEFAULT: `'Resources/agents'`
`description`	Description of the deployment. TYPE: `str \| None` DEFAULT: `None`
`resources`	Resources to be allocated for the predictor. TYPE: `PredictorResources \| dict \| None` DEFAULT: `None`
`inference_logger`	Inference logger configuration. TYPE: `InferenceLogger \| dict \| str \| None` DEFAULT: `None`
`inference_batcher`	Inference batcher configuration. TYPE: `InferenceBatcher \| dict \| None` DEFAULT: `None`
`api_protocol`	API protocol to be enabled in the deployment (i.e., 'REST' or 'GRPC'). TYPE: `str \| None` DEFAULT: `IE.API_PROTOCOL_REST`
`scaling_configuration`	Scaling configuration for the predictor. TYPE: `PredictorScalingConfig \| dict \| None` DEFAULT: `None`
`tracing`	Tracing configuration for the deployment. TYPE: `DeploymentTracingConfig \| dict \| None` DEFAULT: `None`
`git_url`	Optional Git repository URL. When set, `entry` is treated as a relative path inside the repository and is not uploaded. TYPE: `str \| None` DEFAULT: `None`
`git_provider`	Git provider for git-backed agent deployments. TYPE: `str \| None` DEFAULT: `None`
`git_branch`	Optional branch to clone for git-backed agent deployments. TYPE: `str \| None` DEFAULT: `None`
`git_auto_redeploy`	Roll the agent to the branch HEAD whenever a new commit is pushed. Only valid together with `git_url`. The running agent keeps serving until the new version is ready. TYPE: `bool` DEFAULT: `False`
`knative_mode`	Whether to deploy in KServe Knative mode. `None` (default) lets the backend decide: every deployment defaults to Knative mode unless it is a vLLM deployment; on an update, `None` keeps the deployment's current mode. Standard mode does not scale to zero (minimum one instance). It autoscales on a CPU or memory metric between `min_instances` and `max_instances` (default: CPU at 80% up to the cluster maximum), and runs a fixed replica count without autoscaler when `min_instances == max_instances` (the default for LLM deployments). TYPE: `bool \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`Deployment`	The deployment metadata object.

RAISES	DESCRIPTION
`ValueError`	If `entry` is neither a `.py` file nor a directory with `pyproject.toml`, or if `name`/`environment` contain characters outside `[A-Za-z0-9_-]`.
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.

get_deployment #

get_deployment(name: str = None) -> Deployment | None

Get a deployment by name from Model Serving.

Example

# login and get Hopsworks Model Serving handle using .login() and .get_model_serving()

# get a deployment by name
my_deployment = ms.get_deployment('deployment_name')

Getting a deployment from Model Serving means getting its metadata handle so you can subsequently operate on it (e.g., start or stop).

PARAMETER	DESCRIPTION
`name`	Name of the deployment to get. TYPE: `str` DEFAULT: `None`

RETURNS	DESCRIPTION
`Deployment \| None`	The deployment metadata object or `None` if it does not exist.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If unable to retrieve deployment from model serving.

get_deployment_by_id #

get_deployment_by_id(id: int) -> Deployment | None

Get a deployment by id from Model Serving.

Getting a deployment from Model Serving means getting its metadata handle so you can subsequently operate on it (e.g., start or stop).

Example

# login and get Hopsworks Model Serving handle using .login() and .get_model_serving()

# get a deployment by id
my_deployment = ms.get_deployment_by_id(1)

PARAMETER	DESCRIPTION
`id`	Id of the deployment to get. TYPE: `int`

RETURNS	DESCRIPTION
`Deployment \| None`	The deployment metadata object or `None` if it does not exist.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If unable to retrieve deployment from model serving.

get_deployments #

get_deployments(
    model: Model = None, status: str = None
) -> list[Deployment]

Get all deployments from model serving.

Example

# login into Hopsworks using hopsworks.login()

# get Hopsworks Model Registry handle
mr = project.get_model_registry()

# get Hopsworks Model Serving handle
ms = project.get_model_serving()

# retrieve the trained model you want to deploy
my_model = mr.get_model("my_model", version=1)

list_deployments = ms.get_deployment(my_model)

for deployment in list_deployments:
    print(deployment.get_state())

PARAMETER	DESCRIPTION
`model`	Filter by model served in the deployments TYPE: `Model` DEFAULT: `None`
`status`	Filter by status of the deployments TYPE: `str` DEFAULT: `None`

RETURNS	DESCRIPTION
`list[Deployment]`	A list of deployments.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If unable to retrieve deployments from model serving.

get_inference_endpoints #

get_inference_endpoints() -> list[InferenceEndpoint]

Get all inference endpoints available in the current project.

RETURNS	DESCRIPTION
`list[InferenceEndpoint]`	Inference endpoints for model inference

get_vllm_image_tags #

get_vllm_image_tags(
    variant: str = PREDICTOR.VLLM_VARIANT_VLLM,
) -> list[str]

Get the vLLM runtime image tags a cluster administrator advertises.

Use this to pick a valid vllm_image_tag when creating an LLM deployment. Tags come back newest first, so the first one is what a deployment gets when no tag is given. A deployment keeps the tag it was created with even if that tag stops being advertised, so this list can be missing the tag an existing deployment uses.

Example

# login and get Hopsworks Model Serving handle using .login() and .get_model_serving()

# list the tags this cluster advertises for standard vLLM
image_tags = ms.get_vllm_image_tags()

PARAMETER	DESCRIPTION
`variant`	The vLLM variant to list tags for, `VLLM` or `VLLM_OMNI`. TYPE: `str` DEFAULT: `PREDICTOR.VLLM_VARIANT_VLLM`

RETURNS	DESCRIPTION
`list[str]`	Advertised image tags, newest first. Empty if the variant advertises none.

hsml.model_serving #

ModelServing #

project_id property #

project_name property #

project_path property #

create_deployment #

create_endpoint #

create_feature_view_predictor #

create_predictor #

create_transformer #

deploy_agent #

get_deployment #

get_deployment_by_id #

get_deployments #

get_inference_endpoints #

get_vllm_image_tags #

project_id `property` #

project_name `property` #

project_path `property` #