Package com.logicalclocks.hsfs
Class FeatureViewBase<T extends FeatureViewBase,T3 extends FeatureStoreBase<T4>,T4 extends QueryBase,T5>
java.lang.Object
com.logicalclocks.hsfs.FeatureViewBase<T,T3,T4,T5>
- Direct Known Subclasses:
FeatureView,FeatureView,FeatureView,FeatureView
public abstract class FeatureViewBase<T extends FeatureViewBase,T3 extends FeatureStoreBase<T4>,T4 extends QueryBase,T5>
extends Object
-
Field Summary
FieldsModifier and TypeFieldDescriptionprotected Stringprotected Integerprotected List<TrainingDatasetFeature>protected T3protected FeatureViewApiprotected Integerprotected static final org.slf4j.Loggerprotected Stringprotected T4protected TagsApiprotected Stringprotected VectorServerprotected Integer -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionvoidAdd name/value tag to the feature view.voidaddTrainingDatasetTag(Integer version, String name, Object value) Add name/value tag to the training dataset.voidCloses the ExecutorService and JDBC DataSource used to retrieve feature vectors from the online feature store.createTrainingData(String startTime, String endTime, String description, DataFormat dataFormat) Create the metadata for a training dataset and save the corresponding training data into `location`.createTrainingData(String startTime, String endTime, String description, DataFormat dataFormat, Boolean coalesce, DataSource dataSource, Long seed, StatisticsConfig statisticsConfig, Map<String, String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter) Create the metadata for a training dataset and save the corresponding training data into `location`.createTrainingData(String startTime, String endTime, String description, DataFormat dataFormat, Boolean coalesce, StorageConnector storageConnector, String location, Long seed, StatisticsConfig statisticsConfig, Map<String, String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter) Deprecated.createTrainTestSplit(Float testSize, String trainStart, String trainEnd, String testStart, String testEnd, String description, DataFormat dataFormat) Create the metadata for a training dataset and save the corresponding training data into `location`.createTrainTestSplit(Float testSize, String trainStart, String trainEnd, String testStart, String testEnd, String description, DataFormat dataFormat, Boolean coalesce, DataSource dataSource, Long seed, StatisticsConfig statisticsConfig, Map<String, String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter) Create the metadata for a training dataset and save the corresponding training data into `location`.createTrainTestSplit(Float testSize, String trainStart, String trainEnd, String testStart, String testEnd, String description, DataFormat dataFormat, Boolean coalesce, StorageConnector storageConnector, String location, Long seed, StatisticsConfig statisticsConfig, Map<String, String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter) Deprecated.createTrainValidationTestSplit(Float validationSize, Float testSize, String trainStart, String trainEnd, String validationStart, String validationEnd, String testStart, String testEnd, String description, DataFormat dataFormat) Create the metadata for a training dataset and save the corresponding training data into `location`.createTrainValidationTestSplit(Float validationSize, Float testSize, String trainStart, String trainEnd, String validationStart, String validationEnd, String testStart, String testEnd, String description, DataFormat dataFormat, Boolean coalesce, DataSource dataSource, Long seed, StatisticsConfig statisticsConfig, Map<String, String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter) Create the metadata for a training dataset and save the corresponding training data into `location`.createTrainValidationTestSplit(Float validationSize, Float testSize, String trainStart, String trainEnd, String validationStart, String validationEnd, String testStart, String testEnd, String description, DataFormat dataFormat, Boolean coalesce, StorageConnector storageConnector, String location, Long seed, StatisticsConfig statisticsConfig, Map<String, String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter) Deprecated.voiddelete()Delete current feature view, all associated metadata and training data.voidDelete current feature view, all associated metadata and training data.voidDelete a tag of the feature view.voiddeleteTrainingDatasetTag(Integer version, String name) Delete a tag of the training dataset.getFeatureVector(Map<String, Object> entry) Returns assembled feature vector from online feature store.getFeatureVector(Map<String, Object> entry, boolean external) Returns assembled feature vector from online feature store.<T> TgetFeatureVectorObject(Map<String, Object> entry, boolean external, Class<T> returnType) Returns assembled feature vector from online feature store (as Object).<T> TgetFeatureVectorObject(Map<String, Object> entry, Class<T> returnType) Returns assembled feature vector from online feature store (as Object).getFeatureVectors(Map<String, List<Object>> entry) Returns assembled feature vectors in batches from online feature store.getFeatureVectors(Map<String, List<Object>> entry, boolean external) Returns assembled feature vectors in batches from online feature store.Set of primary key names that is used as keys in input dict object for `get_serving_vector` method.Get a single tag value of the feature view.getTags()Get all tags of the feature view.getTrainingDatasetTag(Integer version, String name) Get a single tag value of the training dataset.getTrainingDatasetTags(Integer version) Get all tags of the training dataset.voidinitBatchScoring(Integer trainingDatasetVersion) Initialise feature view to retrieve feature vector from offline feature store.voidInitialise feature view to retrieve feature vector from online feature store.voidinitServing(Boolean batch) Initialise feature view to retrieve feature vector from online feature store.voidinitServing(Boolean batch, Boolean external) Initialise feature view to retrieve feature vector from online feature store.protected voidvalidateTrainTestSplit(Float testSize, String trainEnd, String testStart) protected voidvalidateTrainValidationTestSplit(Float validationSize, Float testSize, String trainEnd, String validationStart, String validationEnd, String testStart)
-
Field Details
-
featureStore
-
id
-
name
-
version
-
description
-
features
-
query
-
labels
-
type
-
LOGGER
protected static final org.slf4j.Logger LOGGER -
featureViewApi
-
tagsApi
-
vectorServer
-
extraFilterVersion
-
-
Constructor Details
-
FeatureViewBase
public FeatureViewBase()
-
-
Method Details
-
createTrainingData
public Integer createTrainingData(String startTime, String endTime, String description, DataFormat dataFormat) throws IOException, FeatureStoreException, ParseException Create the metadata for a training dataset and save the corresponding training data into `location`. The training data can be retrieved by calling `feature_view.getTrainingData()`.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // create training dataset String startTime = "20220101000000"; String endTime = "20220606235959"; String description = "demo training dataset": fv.createTrainingData(startTime, endTime, description, DataFormat.CSV);- Parameters:
startTime- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.endTime- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.description- A string describing the contents of the training dataset to improve discoverability for Data Scientists.dataFormat- The data format used to save the training dataset.- Returns:
- Integer Training dataset version.
- Throws:
FeatureStoreException- If Client is not connected to Hopsworks and/or unable to identify format of the provided `startTime`/`endTime` date formats.IOException- Generic IO exception.ParseException- In case it's unable to parse provided `startTime`/`endTime` strings to date types.
-
createTrainingData
@Deprecated public Integer createTrainingData(String startTime, String endTime, String description, DataFormat dataFormat, Boolean coalesce, StorageConnector storageConnector, String location, Long seed, StatisticsConfig statisticsConfig, Map<String, String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter) throws IOException, FeatureStoreException, ParseExceptionDeprecated.Create the metadata for a training dataset and save the corresponding training data into `location`. The training data can be retrieved by calling `featureView.getTrainingData()`.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // create training dataset String startTime = "20220101000000"; String endTime = "20220606235959"; String description = "demo training dataset": String location = ""; StatisticsConfig statisticsConfig = new StatisticsConfig(true, true, true, true) fv.createTrainingData(startTime, endTime, description, DataFormat.CSV, true, location, statisticsConfig);- Parameters:
startTime- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.endTime- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.description- A string describing the contents of the training dataset to improve discoverability for Data Scientists.dataFormat- The data format used to save the training dataset.coalesce- If true the training dataset data will be coalesced into a single partition before writing. The resulting training dataset will be a single file per split.storageConnector- Storage connector defining the sink location for the training dataset. If `null` is provided and materializes training dataset on HopsFS.location- Path to complement the sink storage connector with, e.g if the storage connector points to an S3 bucket, this path can be used to define a sub-directory inside the bucket to place the training dataset. If empty string is provided `""`, saving the training dataset at the root defined by the storage connector.seed- Define a seed to create the random splits with, in order to guarantee reproducability,statisticsConfig- A configuration object, to generally enable descriptive statistics computation for this feature group, `"correlations`" to turn on feature correlation computation, `"histograms"` to compute feature value frequencies and `"exact_uniqueness"` to compute uniqueness, distinctness and entropy. The values should be booleans indicating the setting. To fully turn off statistics computation pass `statisticsConfig=null`.writeOptions- Additional write options as key-value pairs.extraFilterLogic- Additional filters (set of Filter objects) to be attached to the training dataset. The filters will be also applied in `getBatchData`.extraFilter- Additional filter to be attached to the training dataset. The filter will be also applied in `getBatchData`.- Returns:
- Integer Training dataset version.
- Throws:
FeatureStoreException- If Client is not connected to Hopsworks and/or unable to identify format of the provided `startTime`/`endTime` date formats.IOException- Generic IO exception.ParseException- In case it's unable to parse provided `startTime`/`endTime` strings to date types.
-
createTrainingData
public Integer createTrainingData(String startTime, String endTime, String description, DataFormat dataFormat, Boolean coalesce, DataSource dataSource, Long seed, StatisticsConfig statisticsConfig, Map<String, String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter) throws IOException, FeatureStoreException, ParseExceptionCreate the metadata for a training dataset and save the corresponding training data into `location`. The training data can be retrieved by calling `featureView.getTrainingData()`.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // create training dataset String startTime = "20220101000000"; String endTime = "20220606235959"; String description = "demo training dataset"; DataSource dataSource = fs.getDataSource("my_datasource"); dataSource.setPath("test/path"); StatisticsConfig statisticsConfig = new StatisticsConfig(true, true, true, true); fv.createTrainingData(startTime, endTime, description, DataFormat.CSV, true, dataSource, null, statisticsConfig, null, null, null);- Parameters:
startTime- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.endTime- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.description- A string describing the contents of the training dataset to improve discoverability for Data Scientists.dataFormat- The data format used to save the training dataset.coalesce- If true the training dataset data will be coalesced into a single partition before writing. The resulting training dataset will be a single file per split.dataSource- Data source defining the sink location for the training dataset. If `null` is provided and materializes training dataset on HopsFS.seed- Define a seed to create the random splits with, in order to guarantee reproducability,statisticsConfig- A configuration object, to generally enable descriptive statistics computation for this feature group, `"correlations`" to turn on feature correlation computation, `"histograms"` to compute feature value frequencies and `"exact_uniqueness"` to compute uniqueness, distinctness and entropy. The values should be booleans indicating the setting. To fully turn off statistics computation pass `statisticsConfig=null`.writeOptions- Additional write options as key-value pairs.extraFilterLogic- Additional filters (set of Filter objects) to be attached to the training dataset. The filters will be also applied in `getBatchData`.extraFilter- Additional filter to be attached to the training dataset. The filter will be also applied in `getBatchData`.- Returns:
- Integer Training dataset version.
- Throws:
FeatureStoreException- If Client is not connected to Hopsworks and/or unable to identify format of the provided `startTime`/`endTime` date formats.IOException- Generic IO exception.ParseException- In case it's unable to parse provided `startTime`/`endTime` strings to date types.
-
createTrainTestSplit
public Integer createTrainTestSplit(Float testSize, String trainStart, String trainEnd, String testStart, String testEnd, String description, DataFormat dataFormat) throws IOException, FeatureStoreException, ParseException Create the metadata for a training dataset and save the corresponding training data into `location`. The training data is split into train and test set at random or according to time ranges. The training data can be retrieved by calling `featureView.getTrainTestSplit` method.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // create training dataset based on time split String trainStart = "20220101000000"; String trainEnd = "20220630235959"; String testStart = "20220701000000"; String testEnd = "20220830235959"; String description = "demo training dataset": fv.createTrainTestSplit(null, trainStart, trainEnd, testStart, testEnd, description, DataFormat.CSV); // or based on random split fv.createTrainTestSplit(30, null, null, null, null, description, DataFormat.CSV);- Parameters:
testSize- Size of test set.trainStart- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.trainEnd- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.testStart- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.testEnd- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.description- A string describing the contents of the training dataset to improve discoverability for Data Scientists.dataFormat- The data format used to save the training dataset.- Returns:
- Integer Training dataset version
- Throws:
FeatureStoreException- If Client is not connected to Hopsworks and/or unable to identify format of the provided date strings to date formats.IOException- Generic IO exception.ParseException- In case it's unable to parse provided date strings to date types.
-
createTrainTestSplit
@Deprecated public Integer createTrainTestSplit(Float testSize, String trainStart, String trainEnd, String testStart, String testEnd, String description, DataFormat dataFormat, Boolean coalesce, StorageConnector storageConnector, String location, Long seed, StatisticsConfig statisticsConfig, Map<String, String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter) throws IOException, FeatureStoreException, ParseExceptionDeprecated.Create the metadata for a training dataset and save the corresponding training data into `location`. The training data is split into train and test set at random or according to time ranges. The training data can be retrieved by calling `featureView.getTrainTestSplit` method.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // create training dataset based on time split String trainStart = "20220101000000"; String trainEnd = "20220630235959"; String testStart = "20220701000000"; String testEnd = "20220830235959"; String description = "demo training dataset": StatisticsConfig statisticsConfig = new StatisticsConfig(true, true, true, true) Map<String, String> writeOptions = new HashMap<String, String>() {{ put("header", "true"); put("delimiter", ",")} }; // define extra filters Filter leftFtFilter = new Filter(); leftFtFilter.setFeature(new Feature("left_ft_name")); leftFtFilter.setValue("400"); leftFtFilter.setCondition(SqlFilterCondition.EQUALS); Filter rightFtFilter = new Filter(); rightFtFilter.setFeature(new Feature("right_ft_name")); rightFtFilter.setValue("50"); rightFtFilter.setCondition(SqlFilterCondition.EQUALS); FilterLogic extraFilterLogic = new FilterLogic(SqlFilterLogic.AND, leftFtFilter, rightFtFilter); Filter extraFilter = new Filter(); extraFilter.setFeature(new Feature("ft_name")); extraFilter.setValue("100"); extraFilter.setCondition(SqlFilterCondition.GREATER_THAN); // create training data fv.createTrainTestSplit(null, null, trainStart, trainEnd, testStart, testEnd, description, DataFormat.CSV, coalesce, storageConnector, location, seed, statisticsConfig, writeOptions, extraFilterLogic, extraFilter); // or based on random split fv.createTrainTestSplit(20, 10, null, null, null, null, description, DataFormat.CSV, coalesce, storageConnector, location, seed, statisticsConfig, writeOptions, extraFilterLogic, extraFilter);- Parameters:
testSize- Size of test set.trainStart- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.trainEnd- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.testStart- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.testEnd- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.description- A string describing the contents of the training dataset to improve discoverability for Data Scientists.dataFormat- The data format used to save the training dataset.coalesce- If true the training dataset data will be coalesced into a single partition before writing. The resulting training dataset will be a single file per split.storageConnector- Storage connector defining the sink location for the training dataset. If `null` is provided and materializes training dataset on HopsFS.location- Path to complement the sink storage connector with, e.g if the storage connector points to an S3 bucket, this path can be used to define a sub-directory inside the bucket to place the training dataset. If empty string is provided `""`, saving the training dataset at the root defined by the storage connector.seed- Define a seed to create the random splits with, in order to guarantee reproducability,statisticsConfig- A configuration object, to generally enable descriptive statistics computation for this feature group, `"correlations`" to turn on feature correlation computation, `"histograms"` to compute feature value frequencies and `"exact_uniqueness"` to compute uniqueness, distinctness and entropy. The values should be booleans indicating the setting. To fully turn off statistics computation pass `statisticsConfig=null`.writeOptions- Additional write options as key-value pairs.extraFilterLogic- Additional filters (set of Filter objects) to be attached to the training dataset. The filters will be also applied in `getBatchData`.extraFilter- Additional filter to be attached to the training dataset. The filter will be also applied in `getBatchData`.- Returns:
- Integer Training dataset version.
- Throws:
FeatureStoreException- If Client is not connected to Hopsworks and/or unable to identify format of the provided date strings to date formats.IOException- Generic IO exception.ParseException- In case it's unable to parse provided date strings to date types.
-
createTrainTestSplit
public Integer createTrainTestSplit(Float testSize, String trainStart, String trainEnd, String testStart, String testEnd, String description, DataFormat dataFormat, Boolean coalesce, DataSource dataSource, Long seed, StatisticsConfig statisticsConfig, Map<String, String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter) throws IOException, FeatureStoreException, ParseExceptionCreate the metadata for a training dataset and save the corresponding training data into `location`. The training data is split into train and test set at random or according to time ranges. The training data can be retrieved by calling `featureView.getTrainTestSplit` method.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // create training dataset based on time split String trainStart = "20220101000000"; String trainEnd = "20220630235959"; String testStart = "20220701000000"; String testEnd = "20220830235959"; String description = "demo training dataset": StatisticsConfig statisticsConfig = new StatisticsConfig(true, true, true, true) Map<String, String> writeOptions = new HashMap<String, String>() {{ put("header", "true"); put("delimiter", ",")} }; // define extra filters Filter leftFtFilter = new Filter(); leftFtFilter.setFeature(new Feature("left_ft_name")); leftFtFilter.setValue("400"); leftFtFilter.setCondition(SqlFilterCondition.EQUALS); Filter rightFtFilter = new Filter(); rightFtFilter.setFeature(new Feature("right_ft_name")); rightFtFilter.setValue("50"); rightFtFilter.setCondition(SqlFilterCondition.EQUALS); FilterLogic extraFilterLogic = new FilterLogic(SqlFilterLogic.AND, leftFtFilter, rightFtFilter); Filter extraFilter = new Filter(); extraFilter.setFeature(new Feature("ft_name")); extraFilter.setValue("100"); extraFilter.setCondition(SqlFilterCondition.GREATER_THAN); // create training data fv.createTrainTestSplit(null, null, trainStart, trainEnd, testStart, testEnd, description, DataFormat.CSV, coalesce, dataSource, seed, statisticsConfig, writeOptions, extraFilterLogic, extraFilter); // or based on random split fv.createTrainTestSplit(20, 10, null, null, null, null, description, DataFormat.CSV, coalesce, dataSource, seed, statisticsConfig, writeOptions, extraFilterLogic, extraFilter);- Parameters:
testSize- Size of test set.trainStart- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.trainEnd- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.testStart- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.testEnd- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.description- A string describing the contents of the training dataset to improve discoverability for Data Scientists.dataFormat- The data format used to save the training dataset.coalesce- If true the training dataset data will be coalesced into a single partition before writing. The resulting training dataset will be a single file per split.dataSource- Data source defining the sink location for the training dataset. If `null` is provided and materializes training dataset on HopsFS.seed- Define a seed to create the random splits with, in order to guarantee reproducability,statisticsConfig- A configuration object, to generally enable descriptive statistics computation for this feature group, `"correlations`" to turn on feature correlation computation, `"histograms"` to compute feature value frequencies and `"exact_uniqueness"` to compute uniqueness, distinctness and entropy. The values should be booleans indicating the setting. To fully turn off statistics computation pass `statisticsConfig=null`.writeOptions- Additional write options as key-value pairs.extraFilterLogic- Additional filters (set of Filter objects) to be attached to the training dataset. The filters will be also applied in `getBatchData`.extraFilter- Additional filter to be attached to the training dataset. The filter will be also applied in `getBatchData`.- Returns:
- Integer Training dataset version.
- Throws:
FeatureStoreException- If Client is not connected to Hopsworks and/or unable to identify format of the provided date strings to date formats.IOException- Generic IO exception.ParseException- In case it's unable to parse provided date strings to date types.
-
createTrainValidationTestSplit
public Integer createTrainValidationTestSplit(Float validationSize, Float testSize, String trainStart, String trainEnd, String validationStart, String validationEnd, String testStart, String testEnd, String description, DataFormat dataFormat) throws IOException, FeatureStoreException, ParseException Create the metadata for a training dataset and save the corresponding training data into `location`. The training data is split into train, validation, and test set at random or according to time range. The training data can be retrieved by calling `featureView.getTrainValidationTestSplit`.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // create training dataset based on time split String trainStart = "20220101000000"; String trainEnd = "20220630235959"; String validationStart = "20220701000000"; String validationEnd = "20220830235959"; String testStart = "20220901000000"; String testEnd = "20220931235959"; String description = "demo training dataset": fv.createTrainTestSplit(null, null, trainStart, trainEnd, validationStart, validationEnd, testStart, testEnd, description, DataFormat.CSV); // or based on random split fv.createTrainTestSplit(20, 10, null, null, null, null, null, null, description, DataFormat.CSV);- Parameters:
validationSize- Size of validation set.testSize- Size of test set.trainStart- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.trainEnd- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.validationStart- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.validationEnd- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.testStart- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.testEnd- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.description- A string describing the contents of the training dataset to improve discoverability for Data Scientists.dataFormat- The data format used to save the training dataset.- Returns:
- Integer Training dataset version.
- Throws:
FeatureStoreException- If Client is not connected to Hopsworks and/or unable to identify format of the provided date strings to date formats.IOException- Generic IO exception.ParseException- In case it's unable to parse provided date strings to date types.
-
createTrainValidationTestSplit
@Deprecated public Integer createTrainValidationTestSplit(Float validationSize, Float testSize, String trainStart, String trainEnd, String validationStart, String validationEnd, String testStart, String testEnd, String description, DataFormat dataFormat, Boolean coalesce, StorageConnector storageConnector, String location, Long seed, StatisticsConfig statisticsConfig, Map<String, String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter) throws IOException, FeatureStoreException, ParseExceptionDeprecated.Create the metadata for a training dataset and save the corresponding training data into `location`. The training data is split into train, validation, and test set at random or according to time range. The training data can be retrieved by calling `feature_view.getTrainValidationTestSplit`.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // create training dataset based on time split String trainStart = "20220101000000"; String trainEnd = "20220630235959"; String validationStart = "20220701000000"; String validationEnd = "20220830235959"; String testStart = "20220901000000"; String testEnd = "20220931235959"; String description = "demo training dataset": StorageConnector.S3Connector storageConnector = fs.getS3Connector("s3Connector"); String location = ""; Long seed = 1234L; Boolean coalesce = true; StatisticsConfig statisticsConfig = new StatisticsConfig(true, true, true, true) Map<String, String> writeOptions = new HashMap<String, String>() {{ put("header", "true"); put("delimiter", ",")} }; // define extra filters Filter leftFtFilter = new Filter(); leftFtFilter.setFeature(new Feature("left_ft_name")); leftFtFilter.setValue("400"); leftFtFilter.setCondition(SqlFilterCondition.EQUALS); Filter rightFtFilter = new Filter(); rightFtFilter.setFeature(new Feature("right_ft_name")); rightFtFilter.setValue("50"); rightFtFilter.setCondition(SqlFilterCondition.EQUALS); FilterLogic extraFilterLogic = new FilterLogic(SqlFilterLogic.AND, leftFtFilter, rightFtFilter); Filter extraFilter = new Filter(); extraFilter.setFeature(new Feature("ft_name")); extraFilter.setValue("100"); extraFilter.setCondition(SqlFilterCondition.GREATER_THAN); // create training data fv.createTrainTestSplit(null, null, trainStart, trainEnd, validationStart, validationEnd, testStart, testEnd, description, DataFormat.CSV, coalesce, storageConnector, location, seed, statisticsConfig, writeOptions, extraFilterLogic, extraFilter); // or based on random split fv.createTrainTestSplit(20, 10, null, null, null, null, null, null, description, DataFormat.CSV, coalesce, storageConnector, location, seed, statisticsConfig, writeOptions, extraFilterLogic, extraFilter);- Parameters:
validationSize- Size of validation set.testSize- Size of test set.trainStart- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.trainEnd- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.validationStart- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.validationEnd- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.testStart- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.testEnd- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.description- A string describing the contents of the training dataset to improve discoverability for Data Scientists.dataFormat- The data format used to save the training dataset.coalesce- If true the training dataset data will be coalesced into a single partition before writing. The resulting training dataset will be a single file per split.storageConnector- Storage connector defining the sink location for the training dataset. If `null` is provided and materializes training dataset on HopsFS.location- Path to complement the sink storage connector with, e.g if the storage connector points to an S3 bucket, this path can be used to define a sub-directory inside the bucket to place the training dataset. If empty string is provided `""`, saving the training dataset at the root defined by the storage connector.seed- Define a seed to create the random splits with, in order to guarantee reproducability,statisticsConfig- A configuration object, to generally enable descriptive statistics computation for this feature group, `"correlations`" to turn on feature correlation computation, `"histograms"` to compute feature value frequencies and `"exact_uniqueness"` to compute uniqueness, distinctness and entropy. The values should be booleans indicating the setting. To fully turn off statistics computation pass `statisticsConfig=null`.writeOptions- Additional write options as key-value pairs.extraFilterLogic- Additional filters (set of Filter objects) to be attached to the training dataset. The filters will be also applied in `getBatchData`.extraFilter- Additional filter to be attached to the training dataset. The filter will be also applied in `getBatchData`.- Returns:
- Integer Training dataset version.
- Throws:
FeatureStoreException- If Client is not connected to Hopsworks and/or unable to identify format of the provided date strings to date formats.IOException- Generic IO exception.ParseException- In case it's unable to parse provided date strings to date types.
-
createTrainValidationTestSplit
public Integer createTrainValidationTestSplit(Float validationSize, Float testSize, String trainStart, String trainEnd, String validationStart, String validationEnd, String testStart, String testEnd, String description, DataFormat dataFormat, Boolean coalesce, DataSource dataSource, Long seed, StatisticsConfig statisticsConfig, Map<String, String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter) throws IOException, FeatureStoreException, ParseExceptionCreate the metadata for a training dataset and save the corresponding training data into `location`. The training data is split into train, validation, and test set at random or according to time range. The training data can be retrieved by calling `feature_view.getTrainValidationTestSplit`.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // create training dataset based on time split String trainStart = "20220101000000"; String trainEnd = "20220630235959"; String validationStart = "20220701000000"; String validationEnd = "20220830235959"; String testStart = "20220901000000"; String testEnd = "20220931235959"; String description = "demo training dataset"; DataSource dataSource = fs.getDataSource("my_datasource"); dataSource.setPath("test/path"); Long seed = 1234L; Boolean coalesce = true; StatisticsConfig statisticsConfig = new StatisticsConfig(true, true, true, true) Map<String, String> writeOptions = new HashMap<String, String>() {{ put("header", "true"); put("delimiter", ",")} }; // define extra filters Filter leftFtFilter = new Filter(); leftFtFilter.setFeature(new Feature("left_ft_name")); leftFtFilter.setValue("400"); leftFtFilter.setCondition(SqlFilterCondition.EQUALS); Filter rightFtFilter = new Filter(); rightFtFilter.setFeature(new Feature("right_ft_name")); rightFtFilter.setValue("50"); rightFtFilter.setCondition(SqlFilterCondition.EQUALS); FilterLogic extraFilterLogic = new FilterLogic(SqlFilterLogic.AND, leftFtFilter, rightFtFilter); Filter extraFilter = new Filter(); extraFilter.setFeature(new Feature("ft_name")); extraFilter.setValue("100"); extraFilter.setCondition(SqlFilterCondition.GREATER_THAN); // create training data fv.createTrainTestSplit(null, null, trainStart, trainEnd, validationStart, validationEnd, testStart, testEnd, description, DataFormat.CSV, coalesce, dataSource, seed, statisticsConfig, writeOptions, extraFilterLogic, extraFilter); // or based on random split fv.createTrainTestSplit(20, 10, null, null, null, null, null, null, description, DataFormat.CSV, coalesce, dataSource, seed, statisticsConfig, writeOptions, extraFilterLogic, extraFilter);- Parameters:
validationSize- Size of validation set.testSize- Size of test set.trainStart- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.trainEnd- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.validationStart- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.validationEnd- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.testStart- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.testEnd- Datetime string. The String should be formatted in one of the following formats `yyyyMMdd`, `yyyyMMddHH`, `yyyyMMddHHmm`, or `yyyyMMddHHmmss`.description- A string describing the contents of the training dataset to improve discoverability for Data Scientists.dataFormat- The data format used to save the training dataset.coalesce- If true the training dataset data will be coalesced into a single partition before writing. The resulting training dataset will be a single file per split.dataSource- Data source defining the sink location for the training dataset. If `null` is provided and materializes training dataset on HopsFS.seed- Define a seed to create the random splits with, in order to guarantee reproducability,statisticsConfig- A configuration object, to generally enable descriptive statistics computation for this feature group, `"correlations`" to turn on feature correlation computation, `"histograms"` to compute feature value frequencies and `"exact_uniqueness"` to compute uniqueness, distinctness and entropy. The values should be booleans indicating the setting. To fully turn off statistics computation pass `statisticsConfig=null`.writeOptions- Additional write options as key-value pairs.extraFilterLogic- Additional filters (set of Filter objects) to be attached to the training dataset. The filters will be also applied in `getBatchData`.extraFilter- Additional filter to be attached to the training dataset. The filter will be also applied in `getBatchData`.- Returns:
- Integer Training dataset version.
- Throws:
FeatureStoreException- If Client is not connected to Hopsworks and/or unable to identify format of the provided date strings to date formats.IOException- Generic IO exception.ParseException- In case it's unable to parse provided date strings to date types.
-
validateTrainTestSplit
protected void validateTrainTestSplit(Float testSize, String trainEnd, String testStart) throws FeatureStoreException - Throws:
FeatureStoreException
-
validateTrainValidationTestSplit
protected void validateTrainValidationTestSplit(Float validationSize, Float testSize, String trainEnd, String validationStart, String validationEnd, String testStart) throws FeatureStoreException - Throws:
FeatureStoreException
-
initServing
public void initServing() throws FeatureStoreException, IOException, SQLException, ClassNotFoundExceptionInitialise feature view to retrieve feature vector from online feature store.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // Initialise feature view serving fv.initServing();- Throws:
FeatureStoreException- In case client is not connected to Hopsworks.IOException- Generic IO exception.SQLException- In case there is online storage (RonDB) access error or other errors.ClassNotFoundException- In case class `com.mysql.jdbc.Driver` can not be found.
-
initServing
public void initServing(Boolean batch) throws FeatureStoreException, IOException, SQLException, ClassNotFoundException Initialise feature view to retrieve feature vector from online feature store.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // Initialise feature view batch serving fv.initServing(true);- Parameters:
batch- Whether to initialise feature view to retrieve feature vectors from the online feature store in batches.- Throws:
FeatureStoreException- In case client is not connected to Hopsworks.IOException- Generic IO exception.SQLException- In case there is online storage (RonDB) access error or other errors.ClassNotFoundException- In case class `com.mysql.jdbc.Driver` can not be found.
-
initServing
public void initServing(Boolean batch, Boolean external) throws FeatureStoreException, IOException, SQLException, ClassNotFoundException Initialise feature view to retrieve feature vector from online feature store.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // Initialise feature view batch serving fv.initServing(true, false);- Parameters:
batch- Whether to initialise feature view to retrieve feature vectors from the online feature store in batches.external- If set to `true`, the connection to the online feature store is established using the same host as for the `host` parameter in the connection object. If set to False, the online feature store storage connector is used which relies on the private IP.- Throws:
FeatureStoreException- In case client is not connected to Hopsworks.IOException- Generic IO exception.SQLException- In case there is online storage (RonDB) access error or other errors.ClassNotFoundException- In case class `com.mysql.jdbc.Driver` can not be found.
-
initBatchScoring
Initialise feature view to retrieve feature vector from offline feature store.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // Initialise feature view batch scoring fv.initBatchScoring(1);- Parameters:
trainingDatasetVersion- Version of training dataset to identify additional filters attached to the training dataset and statistics to use for transformation functions.
-
getFeatureVector
public List<Object> getFeatureVector(Map<String, Object> entry) throws FeatureStoreException, IOException, ClassNotFoundExceptionReturns assembled feature vector from online feature store.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // define primary key values to fetch data from online feature store Map<String, Object> pkMap = new HashMap<String, Object>() { {put("customer_id", 1); put("contract_id" , 100); } }; // get feature vector fv.getFeatureVector(entry);- Parameters:
entry- Fictionary of feature group primary key and values provided by serving application.- Returns:
- List of feature values related to provided primary keys, ordered according to positions of the features in the feature view query.
- Throws:
FeatureStoreException- In case client is not connected to Hopsworks.IOException- Generic IO exception.ClassNotFoundException- In case class `com.mysql.jdbc.Driver` can not be found.
-
getFeatureVector
public List<Object> getFeatureVector(Map<String, Object> entry, boolean external) throws FeatureStoreException, IOException, ClassNotFoundExceptionReturns assembled feature vector from online feature store.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // define primary key values to fetch data from online feature store Map<String, Object> pkMap = new HashMap<String, Object>() { {put("customer_id", 1); put("contract_id" , 100); } }; // get feature vector fv.getFeatureVector(entry, false);- Parameters:
entry- Dictionary of feature group primary key and values provided by serving application.external- If set to true, the connection to the online feature store is established using the same host as for the `host` parameter in the connection object. If set to false, the online feature store storage connector is used which relies on the private IP. Defaults to True if connection to Hopsworks is established from external environment- Returns:
- List of feature values related to provided primary keys, ordered according to positions of the features in the feature view query.
- Throws:
FeatureStoreException- In case client is not connected to Hopsworks.IOException- Generic IO exception.ClassNotFoundException- In case class `com.mysql.jdbc.Driver` can not be found.
-
getFeatureVectors
public List<List<Object>> getFeatureVectors(Map<String, List<Object>> entry) throws SQLException, FeatureStoreException, IOExceptionReturns assembled feature vectors in batches from online feature store.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // define primary key values to fetch data from online feature store Map<String, List<Long>> entry = ...; // get feature vector fv.getFeatureVector(entry);- Parameters:
entry- A list of dictionaries of feature group primary key and values provided by serving application.- Returns:
- List of lists of feature values related to provided primary keys, ordered according to positions of the features in the feature view query.
- Throws:
FeatureStoreException- In case client is not connected to Hopsworks.IOException- Generic IO exception.SQLException- In case there is online storage (RonDB) access error or other errors.
-
getFeatureVectors
public List<List<Object>> getFeatureVectors(Map<String, List<Object>> entry, boolean external) throws SQLException, FeatureStoreException, IOException, ClassNotFoundExceptionReturns assembled feature vectors in batches from online feature store.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // define primary key values to fetch data from online feature store Map<String, List<Long>> entry = ...; // get feature vector fv.getFeatureVectors(entry, false);- Parameters:
entry- A list of dictionaries of feature group primary key and values provided by serving application.external- If set to `true`, the connection to the online feature store is established using the same host as for the `host` parameter in the connection object. If set to False, the online feature store storage connector is used which relies on the private IP.- Returns:
- List of lists of feature values related to provided primary keys, ordered according to positions of this features in the feature view query.
- Throws:
FeatureStoreException- In case client is not connected to Hopsworks.IOException- Generic IO exception.SQLException- In case there is online storage (RonDB) access error or other errors.ClassNotFoundException- In case class `com.mysql.jdbc.Driver` can not be found.
-
getFeatureVectorObject
public <T> T getFeatureVectorObject(Map<String, Object> entry, boolean external, Class<T> returnType) throws FeatureStoreException, IOException, ClassNotFoundException, IllegalAccessException, InstantiationExceptionReturns assembled feature vector from online feature store (as Object).// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // define primary key values to fetch data from online feature store Map<String, Object> pkMap = new HashMap<String, Object>() { {put("customer_id", 1); put("contract_id" , 100); } }; // get feature vector fv.getFeatureVectorObject(entry, false, ReturnType.class);- Parameters:
entry- Dictionary of feature group primary key and values provided by serving application.external- If set to true, the connection to the online feature store is established using the same host as for the `host` parameter in the connection object. If set to false, the online feature store storage connector is used which relies on the private IP. Defaults to True if connection to Hopsworks is established from external environmentreturnType- The type of the returned object. Should match the expected structure of the feature vector. The class should also provide the necessary setter methods to set the values of the feature vector.- Returns:
- an instance of type `returnType` containing the values of the requested feature vector
- Throws:
FeatureStoreException- In case client is not connected to Hopsworks.IOException- Generic IO exception.ClassNotFoundException- In case class `com.mysql.jdbc.Driver` can not be found.IllegalAccessException- If the object of type `returnType` cannot be instantiatedInstantiationException- If the object of type `returnType` cannot be instantiated
-
getFeatureVectorObject
public <T> T getFeatureVectorObject(Map<String, Object> entry, Class<T> returnType) throws FeatureStoreException, InstantiationException, IllegalAccessExceptionReturns assembled feature vector from online feature store (as Object).// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // define primary key values to fetch data from online feature store Map<String, Object> pkMap = new HashMap<String, Object>() { {put("customer_id", 1); put("contract_id" , 100); } }; // get feature vector fv.getFeatureVectorObject(entry, ReturnType.class);- Parameters:
entry- Dictionary of feature group primary key and values provided by serving application.returnType- The type of the returned object. Should match the expected structure of the feature vector. The class should also provide the necessary setter methods to set the values of the feature vector.- Returns:
- an instance of type `returnType` containing the values of the requested feature vector
- Throws:
FeatureStoreException- In case client is not connected to Hopsworks.IllegalAccessException- If the object of type `returnType` cannot be instantiatedInstantiationException- If the object of type `returnType` cannot be instantiated
-
addTag
Add name/value tag to the feature view. A tag consists of a name and value pair. Tag names are unique identifiers across the whole cluster. The value of a tag can be any valid json - primitives, arrays or json objects.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // attach a tag to a feature view JSONObject value = ...; fv.addTag("tag_schema", value);- Parameters:
name- Name of the tagvalue- Value of the tag. The value of a tag can be any valid json - primitives, arrays or json objects- Throws:
FeatureStoreException- If Client is not connected to Hopsworks.IOException- Generic IO exception.
-
getTags
Get all tags of the feature view.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // get tags fv.getTags();- Returns:
Map<String, Object>a map of tag name and values. The value of a tag can be any valid json - primitives, arrays or json objects- Throws:
FeatureStoreException- If Client is not connected to Hopsworks.IOException- Generic IO exception.
-
getTag
Get a single tag value of the feature view.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // get tag fv.getTag("tag_name");- Parameters:
name- name of the tag- Returns:
- Object The value of a tag can be any valid json - primitives, arrays or json objects
- Throws:
FeatureStoreException- If Client is not connected to Hopsworks.IOException- Generic IO exception.
-
deleteTag
Delete a tag of the feature view.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // delete tag fv.deleteTag("tag_name");- Parameters:
name- Name of the tag to be deleted.- Throws:
FeatureStoreException- If Client is not connected to Hopsworks.IOException- Generic IO exception.
-
addTrainingDatasetTag
public void addTrainingDatasetTag(Integer version, String name, Object value) throws FeatureStoreException, IOException Add name/value tag to the training dataset.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // add tag to datasets version 1 in this feature view. JSONObject json = ...; fv.addTrainingDatasetTag(1, "tag_name", json);- Parameters:
version- Training dataset version.name- Name of the tag.value- Value of the tag. The value of a tag can be any valid json - primitives, arrays or json objects.- Throws:
FeatureStoreException- If Client is not connected to Hopsworks.IOException- Generic IO exception.
-
getTrainingDatasetTags
public Map<String,Object> getTrainingDatasetTags(Integer version) throws FeatureStoreException, IOException Get all tags of the training dataset.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // get tags of training dataset version 1 in this feature view. fv.getTrainingDatasetTags(1);- Parameters:
version- Training dataset version.- Returns:
Map<String, Object>A map of tag name and values. The value of a tag can be any valid json - primitives, arrays or json objects- Throws:
FeatureStoreException- If Client is not connected to Hopsworks.IOException- Generic IO exception.
-
getTrainingDatasetTag
public Object getTrainingDatasetTag(Integer version, String name) throws FeatureStoreException, IOException Get a single tag value of the training dataset.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // get tag with name `"demo_name"` of training dataset version 1 in this feature view. fv.getTrainingDatasetTags(1, "demo_name");- Parameters:
version- Training dataset version.name- Name of the tag.- Returns:
- Object The value of a tag can be any valid json - primitives, arrays or json objects.
- Throws:
FeatureStoreException- If Client is not connected to Hopsworks.IOException- Generic IO exception.
-
deleteTrainingDatasetTag
public void deleteTrainingDatasetTag(Integer version, String name) throws FeatureStoreException, IOException Delete a tag of the training dataset.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // delete tag with name `"demo_name"` of training dataset version 1 in this feature view. fv.deleteTrainingDatasetTag(1, "demo_name");- Parameters:
version- Tag version.name- Name of the tag to be deleted.- Throws:
FeatureStoreException- If Client is not connected to Hopsworks.IOException- Generic IO exception.
-
delete
Delete current feature view, all associated metadata and training data.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // delete feature view fv.delete();- Throws:
FeatureStoreException- In case client is not connected to Hopsworks.IOException- Generic IO exception.
-
delete
Delete current feature view, all associated metadata and training data. By default, the feature view will not be deleted if there are models associated to it. Ifforceis set totrue, the feature view is deleted even if there are models associated to it.// get feature store handle FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore(); // get feature view handle FeatureView fv = fs.getFeatureView("fv_name", 1); // delete feature view fv.delete(); // or force delete fv.delete(true);- Parameters:
force- If set totrue, forces deletion of the feature view even if there are models associated to it. Defaults tofalse, in which case the feature view will not be deleted if there are models associated to it and an exception will be raised instead.- Throws:
FeatureStoreException- In case client is not connected to Hopsworks.IOException- Generic IO exception.
-
getPrimaryKeys
public HashSet<String> getPrimaryKeys() throws SQLException, IOException, FeatureStoreException, ClassNotFoundExceptionSet of primary key names that is used as keys in input dict object for `get_serving_vector` method.- Returns:
- Set of serving keys
- Throws:
SQLExceptionIOExceptionFeatureStoreExceptionClassNotFoundException
-
closeVectorServer
public void closeVectorServer()Closes the ExecutorService and JDBC DataSource used to retrieve feature vectors from the online feature store.
-