How To Configure A Transformer#

Introduction#

In this guide, you will learn how to configure a transformer script in a model deployment.

Transformer scripts are used to apply transformations on the model inputs before sending them to the predictor for making predictions using the model. They are user-provided Python scripts (.py or .ipynb) implementing the Transformer class.

A transformer has the following configurable components:

See examples of transformer scripts in the serving example notebooks.

Web UI#

Step 1: Create new deployment#

If you have at least one model already trained and saved in the Model Registry, navigate to the deployments page by clicking on the Deployments tab on the navigation menu on the left.

Once in the deployments page, you can create a new deployment by either clicking on New deployment (if there are no existing deployments) or on Create new deployment it the top-right corner. Both options will open the deployment creation form.

Step 2: Go to advanced options#

A simplified creation form will appear including the most common deployment fields from all available configurations. Transformers are part of the advanced options of a deployment. To navigate to the advanced creation form, click on Advanced options.

Step 3: Select a transformer script#

If the transformer script is already located in Hopsworks, click on From project and navigate through the file system to find your script. Otherwise, you can click on Upload new file to upload the transformer script now.

After selecting the transformer script, you can optionally configure resources and autoscaling for your transformer (see Step 4). Otherwise, click on Create new deployment to create the deployment for your model.

Step 4 (Optional): Other advanced options#

In this page, you can also configure the resources to be allocated for the transformer, as well as the autoscaling parameters to control how the transformer scales based on traffic.

Once you are done with the changes, click on Create new deployment at the bottom of the page to create the deployment for your model.

Code#

Step 1: Connect to Hopsworks#

import hopsworks


project = hopsworks.login()

# get Dataset API instance
dataset_api = project.get_dataset_api()

# get Hopsworks Model Registry handle
mr = project.get_model_registry()

Step 2: Implement transformer script#

class Transformer:
    def __init__(self):
        """Initialization code goes here"""
        # Optional __init__ params: project, deployment, model, async_logger
        pass

    def preprocess(self, inputs):
        """Transform the requests inputs here. The object returned by this method will be used as model input to make predictions."""
        return inputs

    def postprocess(self, outputs):
        """Transform the predictions computed by the model before returning a response"""
        return outputs

class Transformer:
    def __init__(self, project, model):
        # Access the project and model directly
        self.project = project
        self.model_metadata = model

Step 3: Upload the script to your project#

uploaded_file_path = dataset_api.upload(
    "my_transformer.py", "Resources", overwrite=True
)
transformer_script_path = os.path.join(
    "/Projects", project.name, uploaded_file_path
)

Step 4: Define a transformer#

my_transformer = ms.create_transformer(script_file=uploaded_file_path)

# or

from hsml.transformer import Transformer

my_transformer = Transformer(script_file)

Step 5: Create a deployment with the transformer#

Use the transformer parameter to set the transformer configuration when creating the model deployment.

my_model = mr.get_model("my_model", version=1)

my_deployment = my_model.deploy(
    transformer=my_transformer
)

API Reference#

Transformer

Transformer script#

A transformer script is a custom Python script to apply pre/post-processing on the model inputs and outputs. This script is included in the artifact files of the deployment. The script must implement the Transformer class, as shown in Step 2.

Resources#

Resources include the number of replicas for the deployment as well as the resources (i.e., memory, CPU, GPU) to be allocated per replica.

To learn about the different combinations available, see the Resources Guide.

Autoscaling#

The transformer has independent autoscaling from the predictor. Deployments use Knative Pod Autoscaler (KPA) to automatically scale the number of replicas based on traffic, including scale-to-zero.

To learn about the different autoscaling parameters, see the Autoscaling Guide.

Environment variables#

A number of different environment variables is available in the transformer to ease its implementation.

Python environments#

Transformer scripts always run on *-inference-pipeline Python environments. To create a new Python environment see Python Environments.