How To Configure A Transformer#

Introduction#

In this guide, you will learn how to configure a transformer in a deployment.

Transformers are used to apply transformations on the model inputs before sending them to the predictor for making predictions using the model. They run on a built-in Flask server provided by Hopsworks and require a user-provided python script implementing the Transformer class.

A transformer has two configurable components:

See examples of transformer scripts in the serving example notebooks.

GUI#

Step 1: Create new deployment#

If you have at least one model already trained and saved in the Model Registry, navigate to the deployments page by clicking on the Deployments tab on the navigation menu on the left.

Once in the deployments page, you can create a new deployment by either clicking on New deployment (if there are no existing deployments) or on Create new deployment it the top-right corner. Both options will open the deployment creation form.

Step 2: Go to advanced options#

A simplified creation form will appear including the most common deployment fields from all available configurations. Transformers are part of the advanced options of a deployment. To navigate to the advanced creation form, click on Advanced options.

Step 3: Select a transformer script#

Transformers require KServe as the serving platform for the deployment. Make sure that KServe is enabled for this deployment by activating the corresponding checkbox.

Then, if the transformer script is already located in Hopsworks, click on From project and navigate through the file system to find your script. Otherwise, you can click on Upload new file to upload the transformer script now.

After selecting the transformer script, you can optionally configure resource allocation for your transformer (see Step 4). Otherwise, click on Create new deployment to create the deployment for your model.

Step 4 (Optional): Configure resource allocation#

At the end of the page, you can configure the resources to be allocated for the transformer, as well as the minimum and maximum number of replicas to be deployed.

Once you are done with the changes, click on Create new deployment at the bottom of the page to create the deployment for your model.

Code#

Step 1: Connect to Hopsworks#

import hopsworks

project = hopsworks.login()

# get Dataset API instance
dataset_api = project.get_dataset_api()

# get Hopsworks Model Serving handle
ms = project.get_model_serving()

Step 2: Implement transformer script#

class Transformer():
    def __init__(self):
        """ Initialization code goes here"""
        pass

    def preprocess(self, inputs):
        """ Transform the requests inputs here. The object returned by this method will be used as model input to make predictions. """
        return inputs

    def postprocess(self, outputs):
        """ Transform the predictions computed by the model before returning a response """
        return outputs

Step 3: Upload the script to your project#

uploaded_file_path = dataset_api.upload("my_transformer.py", "Resources", overwrite=True)
transformer_script_path = os.path.join("/Projects", project.name, uploaded_file_path)

Step 4: Define a transformer#

my_transformer = ms.create_transformer(script_file=uploaded_file_path)

# or

from hsml.transformer import Transformer

my_transformer = Transformer(script_file)

Step 5: Create a deployment with the transformer#

my_predictor = ms.create_predictor(transformer=my_transformer)
my_deployment = my_predictor.deploy()

# or
my_deployment = ms.create_deployment(my_predictor, transformer=my_transformer)
my_deployment.save()

API Reference#

Transformer

Resources#

Resources include the number of replicas for the deployment as well as the resources (i.e., memory, CPU, GPU) to be allocated per replica. To learn about the different combinations available, see the Resources Guide.

Environment variables#

A number of different environment variables is available in the transformer to ease its implementation.