Upgrade existing clusters on Hopsworks.ai from version 2.2 or older (AWS)#
This guide shows you how to upgrade your existing Hopsworks cluster to a newer version of Hopsworks.
Step 1: Make sure your cluster is running#
It is important that your cluster is Running. Otherwise you will not be able to upgrade. As soon as a new version is available an upgrade notification will appear.
You can proceed by clicking the Upgrade button.
Step 2: Add upgrade permissions to your instance profile#
Note
You can skip this step if you already have the following permissions in your instance profile:
[ "ec2:DescribeVolumes", "ec2:DetachVolume", "ec2:AttachVolume", "ec2:ModifyInstanceAttribute"]
We require extra permissions to be added to the instance profile attached to your cluster to proceed with the upgrade. First to get the name of your instance profile, click on the Details tab as shown below:
Once you get your instance profile name, navigate to AWS management console, then click on Roles and then search for your role name and click on it. Go to the Permissions tab, click on Add inline policy, and then go to the JSON tab. Paste the following snippet, click on Review policy, name it, and click Create policy.
Note
You can restrict the upgrade permissions given to your instance profile. Refer to this guide for more information.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "UpgradePermissions",
"Effect": "Allow",
"Action": [
"ec2:DescribeVolumes",
"ec2:DetachVolume",
"ec2:AttachVolume",
"ec2:ModifyInstanceAttribute"
],
"Resource": "*"
}
]
}
Step 3: Run the upgrade process#
You need to click on Upgrade to start the upgrade process. You will be prompted with the screen shown below to confirm your intention to upgrade:
Note
No need to worry about the following message since this is done already in Step 2
Make sure that your instance profile (hopsworks-doc) includes the following permissions: [ "ec2:DetachVolume", "ec2:AttachVolume", "ec2:ModifyInstanceAttribute" ]
Check the Yes, upgrade cluster checkbox to proceed, then the Upgrade button will be activated as shown below:
Warning
Currently, we only support upgrade for the head node and you will need to recreate your workers once the upgrade is successfully completed.
Depending on how big your current cluster is, the upgrade process may take from 1 hour to a few hours until completion.
Note
We don't delete your old cluster until the upgrade process is successfully completed.
Once the upgrade is completed, you can confirm that you have the new Hopsworks version by checking the Details tab of your cluster as below:
Error handling#
There are two categories of errors that you may encounter during an upgrade. First, a permission error due to missing permission or a misconfigured policy in your instance profile, see Error 1. Second, an error during the upgrade process running on your cluster, see Error 2.
Error 1: Misconfigured upgrade permissions#
During the upgrade process, Hopsworks.ai starts by validating your instance profile permissions to ensure that it includes the required upgrade permissions. If one or more permissions are missing, or if the resource is not set correctly, you will be notified with an error message and a Retry button will appear as shown below:
Update you instance profile accordingly, then click Retry
Error 2: Upgrade process error#
If an error occurs during the upgrade process, you will have the option to rollback to your old cluster as shown below:
Click on Rollback to recover your old cluster before upgrade.
Check the Yes, rollback cluster checkbox to proceed, then the Rollback button will be activated as shown below:
Once the rollback is completed, you will be able to continue working as normal with your old cluster.
Note
The old cluster will be stopped after the rollback. You have to click on the Start button.