Skip to content

Project/Connection#

In Hopsworks a Project is a sandboxed set of users, data, and programs (where data can be shared in a controlled manner between projects).

Each Project can have its own Model Registry. However, it is possible to share Model Registry among projects.

When working with the Model Registry from a programming environment you can connect to a single Hopsworks instance at a time, but it is possible to access multiple Model Registries simultaneously.

A connection to a Hopsworks instance is represented by a Connection object. Its main purpose is to retrieve the API Key if you are connecting from an external environment.

The handle can then be used to retrieve a reference to the Model Registry you want to operate on.

Examples#

Connecting from Hopsworks

import hsml
conn = hsml.connection()
mr = conn.get_model_registry()

Connecting from Python environment

To connect from an external Python environment you can provide the api_key_value directly:

import hsml
conn = hsml.connection(
    host="ec2-13-53-124-128.eu-north-1.compute.amazonaws.com",
    project="demo_ml_admin000",
    hostname_verification=False,
    api_key_value="PFcy3dZ6wLXYglRd.ydcdq5jH878IdG7xlL9lHVqrS8v3sBUqQgyR4xbpUgDnB5ZpYro6O"
    )
mr = conn.get_model_registry()

Alternatively you can pass the API Key as a file or directly:

import hsml
conn = hsml.connection(
    host="ec2-13-53-124-128.eu-north-1.compute.amazonaws.com",
    project="demo_ml_admin000",
    hostname_verification=False,
    api_key_file="modelregistry.key"
    )
mr = conn.get_model_registry()

Sharing a Model Registry#

Connections are on a project-level, however, it is possible to share model registries among projects, so even if you have a connection to one project, you can retrieve a handle to any model registry shared with that project.

To share a model registry, you can follow these steps:

Sharing a Model Registry

  1. Open the project of the model registry that you would like to share on Hopsworks.
  2. Go to the Data Set browser and right click the Models entry.
  3. Click Share with, then select Project and choose the project you wish to share the model registry with.
  4. Select the permissions level that the project user members should have on the model registry and click Share.
  5. Open the project you just shared the model registry with.
  6. Go to the Data Sets browser and there you should see the shared model registry as [project_name_of_shared_model_registry]::Models. Click this entry, you will be asked to accept this shared Dataset, click Accept.
  7. You should now have access to this model registry from the other project.

Sharing a model registry between projects
Sharing a model registry between projects

Accepting a shared model registry
Accepting a shared model registry from a project

Connection Handle#

[source]

Connection#

hsml.connection.Connection(
    host=None,
    port=443,
    project=None,
    region_name="default",
    secrets_store="parameterstore",
    hostname_verification=True,
    trust_store_path=None,
    api_key_file=None,
    api_key_value=None,
)

A Model registry connection object.

The connection is project specific, so you can access the project's own model registry.

This class provides convenience classmethods accessible from the hsml-module:

Connection factory

For convenience, hsml provides a factory method, accessible from the top level module, so you don't have to import the Connection class manually:

import hsml
conn = hsml.connection()

Save API Key as File

To get started quickly, without saving the Hopsworks API in a secret storage, you can simply create a file with the previously created Hopsworks API Key and place it on the environment from which you wish to connect to the Hopsworks Model Registry.

You can then connect by simply passing the path to the key file when instantiating a connection:

    import hsml
    conn = hsml.connection(
        'my_instance',                      # DNS of your Model Registry instance
        443,                                # Port to reach your Hopsworks instance, defaults to 443
        'my_project',                       # Name of your Hopsworks Model Registry project
        api_key_file='modelregistry.key',   # The file containing the API key generated above
        hostname_verification=True)         # Disable for self-signed certificates
    )
    mr = conn.get_model_registry()          # Get the project's default model registry
    ms = conn.get_model_serving()           # Get the project's default model serving

Clients in external clusters need to connect to the Hopsworks Model Registry using an API key. The API key is generated inside the Hopsworks platform, and requires at least the "project", "modelregistry", "dataset.create", "dataset.view", "dataset.delete", "serving" scopes to be able to access a model registry and its model serving. For more information, see the integration guides.

Arguments

  • host Optional[str]: The hostname of the Hopsworks instance, defaults to None.
  • port int: The port on which the Hopsworks instance can be reached, defaults to 443.
  • project Optional[str]: The name of the project to connect to. When running on Hopsworks, this defaults to the project from where the client is run from. Defaults to None.
  • region_name str: The name of the AWS region in which the required secrets are stored, defaults to "default".
  • secrets_store str: The secrets storage to be used, either "secretsmanager", "parameterstore" or "local", defaults to "parameterstore".
  • hostname_verification bool: Whether or not to verify Hopsworks certificate, defaults to True.
  • trust_store_path Optional[str]: Path on the file system containing the Hopsworks certificates, defaults to None.
  • api_key_file Optional[str]: Path to a file containing the API Key, if provided, secrets_store will be ignored, defaults to None.
  • api_key_value Optional[str]: API Key as string, if provided, secrets_store will be ignored, however, this should be used with care, especially if the used notebook or job script is accessible by multiple parties. Defaults toNone`.

Returns

Connection. Connection handle to perform operations on a Hopsworks project.

Methods#

[source]

close#

Connection.close()

Close a connection gracefully.

This will clean up any materialized certificates on the local file system of external environments such as AWS SageMaker.

Usage is recommended but optional.

[source]

connect#

Connection.connect()

Instantiate the connection.

Creating a Connection object implicitly calls this method for you to instantiate the connection. However, it is possible to close the connection gracefully with the close() method, in order to clean up materialized certificates. This might be desired when working on external environments such as AWS SageMaker. Subsequently you can call connect() again to reopen the connection.

Example

import hsml
conn = hsml.connection()
conn.close()
conn.connect()

[source]

get_model_registry#

Connection.get_model_registry(project=None)

Get a reference to a model registry to perform operations on. Defaulting to the project's default model registry. Shared model registries can be retrieved by passing the project argument. Arguments

  • project str: The name of the project that owns the shared model registry, the model registry must be shared with the project the connection was established for, defaults to None.

Returns

ModelRegistry. A model registry handle object to perform operations on.

[source]

get_model_serving#

Connection.get_model_serving()

Get a reference to model serving to perform operations on. Model serving operates on top of a model registry. Connecting to the project's default model registry. Returns

ModelServing. A model serving handle object to perform operations on.