com.logicalclocks.hsfs.beam

Class FeatureStore

java.lang.Object

com.logicalclocks.hsfs.FeatureStoreBase<com.logicalclocks.hsfs.beam.constructor.Query>

com.logicalclocks.hsfs.beam.FeatureStore

public class FeatureStore
extends FeatureStoreBase<com.logicalclocks.hsfs.beam.constructor.Query>

Field Summary

Fields inherited from class com.logicalclocks.hsfs.FeatureStoreBase

DEFAULT_VERSION, featureGroupApi, LOGGER, storageConnectorApi, trainingDatasetApi

Constructor Summary

Constructors

Constructor and Description
FeatureStore()

Method Summary

Modifier and Type	Method and Description
com.logicalclocks.hsfs.beam.StreamFeatureGroup.StreamFeatureGroupBuilder	createStreamFeatureGroup() Create a feature group builder object.
StreamFeatureGroup	createStreamFeatureGroup(@NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, StorageConnector storageConnector, String path) Create a feature group object.
FeatureView	getFeatureView(String name) Get a feature view object from the selected feature store.
FeatureView	getFeatureView(@NonNull String name, @NonNull Integer version) Get a feature view object from the selected feature store.
StreamFeatureGroup	getOrCreateStreamFeatureGroup(@NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, StorageConnector storageConnector, String path) Get a feature group metadata object or create a new one if it doesn't exists.
Object	getStreamFeatureGroup(String name) Get a stream feature group object from the feature store.
StreamFeatureGroup	getStreamFeatureGroup(@NonNull String name, @NonNull Integer version) Get a stream feature group object from the feature store.

Methods inherited from class com.logicalclocks.hsfs.FeatureStoreBase

getAdlsConnector, getBigqueryConnector, getGcsConnector, getHopsFsConnector, getJdbcConnector, getKafkaConnector, getOnlineStorageConnector, getRedshiftConnector, getS3Connector, getSnowflakeConnector, getStorageConnector, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

FeatureStore

public FeatureStore()

Method Detail

createStreamFeatureGroup

public com.logicalclocks.hsfs.beam.StreamFeatureGroup.StreamFeatureGroupBuilder createStreamFeatureGroup()

Create a feature group builder object.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();

        // create feature group metadata object
        StreamFeatureGroup streamFeatureGroup = fs.createStreamFeatureGroup()
               .name("documentation_example")
               .version(1)
               .primaryKeys(Collections.singletonList("pk"))
               .eventTime("event_time")
               .onlineEnabled(true)
               .features(features)
               .build();

         // save the feature group metadata object on the feature store
         streamFeatureGroup.save()
 
 

Returns:

StreamFeatureGroup.StreamFeatureGroupBuilder a StreamFeatureGroup builder object.

createStreamFeatureGroup

public StreamFeatureGroup createStreamFeatureGroup(@NonNull
                                                   @NonNull String name,
                                                   Integer version,
                                                   String description,
                                                   Boolean onlineEnabled,
                                                   TimeTravelFormat timeTravelFormat,
                                                   List<String> primaryKeys,
                                                   List<String> partitionKeys,
                                                   String eventTime,
                                                   String hudiPrecombineKey,
                                                   List<Feature> features,
                                                   StatisticsConfig statisticsConfig,
                                                   StorageConnector storageConnector,
                                                   String path)

Description copied from class: FeatureStoreBase

Create a feature group object.

Specified by:

createStreamFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.beam.constructor.Query>

Parameters:

name - the name of the feature group

version - the version of the feature group

description - descrption of the feature group

onlineEnabled - whether the feature group should be online enabled

timeTravelFormat - the data format to use to store the offline data

primaryKeys - list of primary keys

partitionKeys - list of partition keys

eventTime - the feature/column to use as event time

hudiPrecombineKey - if the timeTravelFormat is set to hudi, the feature/column to use as precombine key

features - the list of feature objects if defined explicitly

statisticsConfig - the statistics configuration for the feature group

storageConnector - the storage connector to use to store the offline feature data (Default stored internally in Hopsworks).

path - the path on the storage where to store the feature data.

Returns:

The feature group metadata object.

getOrCreateStreamFeatureGroup

public StreamFeatureGroup getOrCreateStreamFeatureGroup(@NonNull
                                                        @NonNull String name,
                                                        Integer version,
                                                        String description,
                                                        Boolean onlineEnabled,
                                                        TimeTravelFormat timeTravelFormat,
                                                        List<String> primaryKeys,
                                                        List<String> partitionKeys,
                                                        String eventTime,
                                                        String hudiPrecombineKey,
                                                        List<Feature> features,
                                                        StatisticsConfig statisticsConfig,
                                                        StorageConnector storageConnector,
                                                        String path)
                                                 throws IOException,
                                                        FeatureStoreException

Description copied from class: FeatureStoreBase

Get a feature group metadata object or create a new one if it doesn't exists.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        StreamFeatureGroup streamFeatureGroup = featureStore.getOrCreateStreamFeatureGroup(
                 "doc_example",
                 1,
                 "Feature group for example in documentation",
                 true,
                 TimeTravelFormat.HUDI,
                 Collections.singletonList("primary_key"),
                 Collections.singletonList("partition_key"),
                 "event_time",
                 null,
                 features,
                 null,
                 new StatisticsConfig(true, true, true, true),
                 null
         );

         streamFeatureGroup.save()
 
 

Specified by:

getOrCreateStreamFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.beam.constructor.Query>

Parameters:

name - the name of the feature group

version - the version of the feature group

description - descrption of the feature group

onlineEnabled - whether the feature group should be online enabled

timeTravelFormat - the data format to use to store the offline data

primaryKeys - list of primary keys

partitionKeys - list of partition keys

eventTime - the feature/column to use as event time

hudiPrecombineKey - if the timeTravelFormat is set to hudi, the feature/column to use as precombine key

features - the list of feature objects if defined explicitly

statisticsConfig - the statistics configuration for the feature group

storageConnector - the storage connector to use to store the offline feature data (Default stored internally in Hopsworks).

path - the path on the storage where to store the feature data.

Returns:

The feature group metadata object.

Throws:

IOException

FeatureStoreException

getStreamFeatureGroup

public Object getStreamFeatureGroup(String name)
                             throws FeatureStoreException,
                                    IOException

Get a stream feature group object from the feature store.

Getting a stream feature group metadata handle enables to interact with the feature group, such as read the data or use the `Query`-API to perform joins between feature groups and create feature views.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getStreamFeatureGroup("electricity_prices", 1);
 
 

Specified by:

getStreamFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.beam.constructor.Query>

Parameters:

name - the name of the feature group

Returns:

StreamFeatureGroup The stream feature group metadata object.

Throws:

FeatureStoreException - If unable to retrieve feature group from the feature store.

IOException - Generic IO exception.

getStreamFeatureGroup

public StreamFeatureGroup getStreamFeatureGroup(@NonNull
                                                @NonNull String name,
                                                @NonNull
                                                @NonNull Integer version)
                                         throws FeatureStoreException,
                                                IOException

Get a stream feature group object from the feature store.

Getting a stream feature group metadata handle enables to interact with the feature group, such as read the data or use the `Query`-API to perform joins between feature groups and create feature views.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getStreamFeatureGroup("electricity_prices", 1);
 
 

Specified by:

getStreamFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.beam.constructor.Query>

Parameters:

name - the name of the feature group

version - the version of the feature group

Returns:

StreamFeatureGroup The stream feature group metadata object.

Throws:

FeatureStoreException - If unable to retrieve feature group from the feature store.

IOException - Generic IO exception.

getFeatureView

public FeatureView getFeatureView(@NonNull
                                  @NonNull String name,
                                  @NonNull
                                  @NonNull Integer version)
                           throws FeatureStoreException,
                                  IOException

Get a feature view object from the selected feature store.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureView fv = fs.getFeatureView("fv_name", 1);
 
 

Specified by:

getFeatureView in class FeatureStoreBase<com.logicalclocks.hsfs.beam.constructor.Query>

Parameters:

name - Name of the feature view.

version - Version to get.

Returns:

FeatureView The feature view metadata object.

Throws:

FeatureStoreException - If unable to retrieve FeatureView from the feature store.

IOException - Generic IO exception.

getFeatureView

public FeatureView getFeatureView(String name)
                           throws FeatureStoreException,
                                  IOException

Get a feature view object from the selected feature store.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureView fv = fs.getFeatureView("fv_name", 1);
 
 

Specified by:

getFeatureView in class FeatureStoreBase<com.logicalclocks.hsfs.beam.constructor.Query>

Parameters:

name - Name of the feature view.

Returns:

FeatureView The feature view metadata object.

Throws:

FeatureStoreException - If unable to retrieve FeatureView from the feature store.

IOException - Generic IO exception.