Skip to content

Adding and removing workers#

Once you have started a hopsworks cluster you can add and remove workers from the cluster to accommodate your workload.

Adding workers#

If the computation you are running is using all the resources of your Hopsworks cluster you can add workers to your cluster. To add workers to a cluster, go to the Details tab of this cluster and click on Add workers.

Add worker
Add worker

Select the number of workers you want to add (1). Select the type of instance you want the workers to run on (2). Select the local storage size for the workers (3). Click on Next.

Add workers config
Add workers

Review your request and click Add.

Add workers review
Add workers

Managed.hopsworks.ai will start the new workers and you will be able to use them in your cluster as soon as they have finished starting.

Removing workers#

If the load on your Hopsworks cluster is low, you can decide to remove worker nodes from your cluster.

Warning

When removing workers managed.hopsworks.ai will try to select workers that can be removed while interfering as little as possible with any ongoing computation. It will also wait for the workers to be done with their computation before stopping them. But, if this computation lasts too long, the worker may get stopped before the computation properly finish. This could interfere with your ongoing computation.

Note

You can remove all the workers of your cluster. If you do so the cluster will be able to store data but not run any computations. This may affect feature store functionality.

To remove workers from a cluster, go to the Details tab of this cluster and click on Remove workers

Remove worker
Remove workers

For each of the types of instances existing in your cluster select the number of workers you want to remove and click on Next.

Remove workers config
Remove workers

Review your request and click Remove.

Remove workers review
Remove workers

Managed.hopsworks.ai will select the workers corresponding to your criteria which can be stopped with as little interferences as possible with any ongoing computation. It will set them to decommission and stop them when they have finished decommissioning.