com.logicalclocks.hsfs.flink

Class FeatureStore

java.lang.Object

com.logicalclocks.hsfs.FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

com.logicalclocks.hsfs.flink.FeatureStore

public class FeatureStore
extends FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Field Summary

Fields inherited from class com.logicalclocks.hsfs.FeatureStoreBase

DEFAULT_VERSION, featureGroupApi, LOGGER, storageConnectorApi, trainingDatasetApi

Constructor Summary

Constructors

Constructor and Description
FeatureStore()

Method Summary

Modifier and Type	Method and Description
Object	createExternalFeatureGroup()
Object	createFeatureGroup()
Object	createFeatureView()
com.logicalclocks.hsfs.flink.StreamFeatureGroup.StreamFeatureGroupBuilder	createStreamFeatureGroup()
Object	getAdlsConnector(String name)
Object	getBigqueryConnector(String name)
Object	getExternalFeatureGroup(String name) Get a external feature group object with default version `1` from the feature store.
Object	getExternalFeatureGroup(@NonNull String name, @NonNull Integer version) Get a external feature group object from the feature store.
Object	getExternalFeatureGroups(@NonNull String name)
Object	getFeatureGroups(@NonNull String name)
FeatureView	getFeatureView(String name) Get a feature view object from the selected feature store.
FeatureView	getFeatureView(@NonNull String name, @NonNull Integer version) Get a feature view object from the selected feature store.
Object	getGcsConnector(String name)
Object	getHopsFsConnector(String name)
Object	getJdbcConnector(String name)
Object	getKafkaConnector(String name)
Object	getOnlineStorageConnector()
Object	getOrCreateFeatureGroup(String name, Integer version)
Object	getOrCreateFeatureGroup(String name, Integer integer, List<String> primaryKeys, boolean onlineEnabled, String eventTime)
Object	getOrCreateFeatureGroup(String name, Integer version, List<String> primaryKeys, List<String> partitionKeys, boolean onlineEnabled, String eventTime)
Object	getOrCreateFeatureGroup(String name, Integer version, String description, List<String> primaryKeys, List<String> partitionKeys, String hudiPrecombineKey, boolean onlineEnabled, TimeTravelFormat timeTravelFormat, StatisticsConfig statisticsConfig, String topicName, String notificationTopicName, String eventTime)
FeatureView	getOrCreateFeatureView(String name, com.logicalclocks.hsfs.flink.constructor.Query query, Integer version)
FeatureView	getOrCreateFeatureView(String name, com.logicalclocks.hsfs.flink.constructor.Query query, Integer version, String description, List<String> labels)
StreamFeatureGroup	getOrCreateStreamFeatureGroup(String name, Integer version)
StreamFeatureGroup	getOrCreateStreamFeatureGroup(String name, Integer version, List<String> primaryKeys, boolean onlineEnabled, String eventTime)
StreamFeatureGroup	getOrCreateStreamFeatureGroup(String name, Integer version, List<String> primaryKeys, List<String> partitionKeys, boolean onlineEnabled, String eventTime)
StreamFeatureGroup	getOrCreateStreamFeatureGroup(String name, Integer version, String description, List<String> primaryKeys, List<String> partitionKeys, String hudiPrecombineKey, boolean onlineEnabled, StatisticsConfig statisticsConfig, String eventTime)
Object	getRedshiftConnector(String name)
Object	getS3Connector(String name)
Object	getSnowflakeConnector(String name)
StorageConnector	getStorageConnector(String name) Get a list of all versions of an external feature group from the feature store.
StreamFeatureGroup	getStreamFeatureGroup(String name) Get a stream feature group object from the feature store.
StreamFeatureGroup	getStreamFeatureGroup(@NonNull String name, @NonNull Integer version) Get a stream feature group object from the feature store.
TrainingDatasetBase	getTrainingDataset(String name) Get a training dataset object with the default version `1` from the selected feature store.
TrainingDatasetBase	getTrainingDataset(@NonNull String name, @NonNull Integer version) Get a training dataset object from the selected feature store.
Object	getTrainingDatasets(@NonNull String name) Get all versions of a training dataset object from the selected feature store.
Object	sql(String query)

Methods inherited from class com.logicalclocks.hsfs.FeatureStoreBase

toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

FeatureStore

public FeatureStore()

Method Detail

createFeatureGroup

public Object createFeatureGroup()

Specified by:

createFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

getFeatureGroups

public Object getFeatureGroups(@NonNull
                               @NonNull String name)
                        throws FeatureStoreException,
                               IOException

Specified by:

getFeatureGroups in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

FeatureStoreException

IOException

getOrCreateFeatureGroup

public Object getOrCreateFeatureGroup(String name,
                                      Integer version)
                               throws IOException,
                                      FeatureStoreException

Specified by:

getOrCreateFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

IOException

FeatureStoreException

getOrCreateFeatureGroup

public Object getOrCreateFeatureGroup(String name,
                                      Integer integer,
                                      List<String> primaryKeys,
                                      boolean onlineEnabled,
                                      String eventTime)
                               throws IOException,
                                      FeatureStoreException

Specified by:

getOrCreateFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

IOException

FeatureStoreException

getOrCreateFeatureGroup

public Object getOrCreateFeatureGroup(String name,
                                      Integer version,
                                      List<String> primaryKeys,
                                      List<String> partitionKeys,
                                      boolean onlineEnabled,
                                      String eventTime)
                               throws IOException,
                                      FeatureStoreException

Specified by:

getOrCreateFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

IOException

FeatureStoreException

getOrCreateFeatureGroup

public Object getOrCreateFeatureGroup(String name,
                                      Integer version,
                                      String description,
                                      List<String> primaryKeys,
                                      List<String> partitionKeys,
                                      String hudiPrecombineKey,
                                      boolean onlineEnabled,
                                      TimeTravelFormat timeTravelFormat,
                                      StatisticsConfig statisticsConfig,
                                      String topicName,
                                      String notificationTopicName,
                                      String eventTime)

Specified by:

getOrCreateFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

getStreamFeatureGroup

public StreamFeatureGroup getStreamFeatureGroup(String name)
                                         throws FeatureStoreException,
                                                IOException

Get a stream feature group object from the feature store.

Getting a stream feature group metadata handle enables to interact with the feature group, such as read the data or use the `Query`-API to perform joins between feature groups and create feature views.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getStreamFeatureGroup("electricity_prices", 1);
 
 

Specified by:

getStreamFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Parameters:

name - the name of the feature group

Returns:

StreamFeatureGroup The stream feature group metadata object.

Throws:

FeatureStoreException - If unable to retrieve feature group from the feature store.

IOException - Generic IO exception.

getStreamFeatureGroup

public StreamFeatureGroup getStreamFeatureGroup(@NonNull
                                                @NonNull String name,
                                                @NonNull
                                                @NonNull Integer version)
                                         throws FeatureStoreException,
                                                IOException

Get a stream feature group object from the feature store.

Getting a stream feature group metadata handle enables to interact with the feature group, such as read the data or use the `Query`-API to perform joins between feature groups and create feature views.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getStreamFeatureGroup("electricity_prices", 1);
 
 

Specified by:

getStreamFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Parameters:

name - the name of the feature group

version - the version of the feature group

Returns:

StreamFeatureGroup The stream feature group metadata object.

Throws:

FeatureStoreException - If unable to retrieve feature group from the feature store.

IOException - Generic IO exception.

createStreamFeatureGroup

public com.logicalclocks.hsfs.flink.StreamFeatureGroup.StreamFeatureGroupBuilder createStreamFeatureGroup()

Specified by:

createStreamFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

getOrCreateStreamFeatureGroup

public StreamFeatureGroup getOrCreateStreamFeatureGroup(String name,
                                                        Integer version)
                                                 throws IOException,
                                                        FeatureStoreException

Specified by:

getOrCreateStreamFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

IOException

FeatureStoreException

getOrCreateStreamFeatureGroup

public StreamFeatureGroup getOrCreateStreamFeatureGroup(String name,
                                                        Integer version,
                                                        List<String> primaryKeys,
                                                        boolean onlineEnabled,
                                                        String eventTime)
                                                 throws IOException,
                                                        FeatureStoreException

Specified by:

getOrCreateStreamFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

IOException

FeatureStoreException

getOrCreateStreamFeatureGroup

public StreamFeatureGroup getOrCreateStreamFeatureGroup(String name,
                                                        Integer version,
                                                        List<String> primaryKeys,
                                                        List<String> partitionKeys,
                                                        boolean onlineEnabled,
                                                        String eventTime)
                                                 throws IOException,
                                                        FeatureStoreException

Specified by:

getOrCreateStreamFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

IOException

FeatureStoreException

getOrCreateStreamFeatureGroup

public StreamFeatureGroup getOrCreateStreamFeatureGroup(String name,
                                                        Integer version,
                                                        String description,
                                                        List<String> primaryKeys,
                                                        List<String> partitionKeys,
                                                        String hudiPrecombineKey,
                                                        boolean onlineEnabled,
                                                        StatisticsConfig statisticsConfig,
                                                        String eventTime)
                                                 throws IOException,
                                                        FeatureStoreException

Specified by:

getOrCreateStreamFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

IOException

FeatureStoreException

createExternalFeatureGroup

public Object createExternalFeatureGroup()

Specified by:

createExternalFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

createFeatureView

public Object createFeatureView()

Specified by:

createFeatureView in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

getStorageConnector

public StorageConnector getStorageConnector(String name)
                                     throws FeatureStoreException,
                                            IOException

Description copied from class: FeatureStoreBase

Get a list of all versions of an external feature group from the feature store.

Specified by:

getStorageConnector in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Parameters:

name - the name of the feature group

Returns:

ExternalFeatureGroup

Throws:

FeatureStoreException

IOException

getHopsFsConnector

public Object getHopsFsConnector(String name)
                          throws FeatureStoreException,
                                 IOException

Specified by:

getHopsFsConnector in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

FeatureStoreException

IOException

getJdbcConnector

public Object getJdbcConnector(String name)
                        throws FeatureStoreException,
                               IOException

Specified by:

getJdbcConnector in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

FeatureStoreException

IOException

getOnlineStorageConnector

public Object getOnlineStorageConnector()
                                 throws FeatureStoreException,
                                        IOException

Specified by:

getOnlineStorageConnector in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

FeatureStoreException

IOException

getGcsConnector

public Object getGcsConnector(String name)
                       throws FeatureStoreException,
                              IOException

Specified by:

getGcsConnector in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

FeatureStoreException

IOException

getS3Connector

public Object getS3Connector(String name)
                      throws FeatureStoreException,
                             IOException

Specified by:

getS3Connector in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

FeatureStoreException

IOException

getRedshiftConnector

public Object getRedshiftConnector(String name)
                            throws FeatureStoreException,
                                   IOException

Specified by:

getRedshiftConnector in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

FeatureStoreException

IOException

getSnowflakeConnector

public Object getSnowflakeConnector(String name)
                             throws FeatureStoreException,
                                    IOException

Specified by:

getSnowflakeConnector in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

FeatureStoreException

IOException

getAdlsConnector

public Object getAdlsConnector(String name)
                        throws FeatureStoreException,
                               IOException

Specified by:

getAdlsConnector in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

FeatureStoreException

IOException

getKafkaConnector

public Object getKafkaConnector(String name)
                         throws FeatureStoreException,
                                IOException

Specified by:

getKafkaConnector in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

FeatureStoreException

IOException

getBigqueryConnector

public Object getBigqueryConnector(String name)
                            throws FeatureStoreException,
                                   IOException

Specified by:

getBigqueryConnector in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

FeatureStoreException

IOException

getExternalFeatureGroups

public Object getExternalFeatureGroups(@NonNull
                                       @NonNull String name)
                                throws FeatureStoreException,
                                       IOException

Specified by:

getExternalFeatureGroups in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

FeatureStoreException

IOException

sql

public Object sql(String query)

Specified by:

sql in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

getTrainingDataset

public TrainingDatasetBase getTrainingDataset(@NonNull
                                              @NonNull String name,
                                              @NonNull
                                              @NonNull Integer version)
                                       throws FeatureStoreException,
                                              IOException

Description copied from class: FeatureStoreBase

Get a training dataset object from the selected feature store.

Specified by:

getTrainingDataset in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Parameters:

name - name of the training dataset

version - version to get

Returns:

TrainingDataset

Throws:

FeatureStoreException

IOException

getTrainingDataset

public TrainingDatasetBase getTrainingDataset(String name)
                                       throws FeatureStoreException,
                                              IOException

Description copied from class: FeatureStoreBase

Get a training dataset object with the default version `1` from the selected feature store.

Specified by:

getTrainingDataset in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Parameters:

name - name of the training dataset

Returns:

TrainingDataset

Throws:

FeatureStoreException

IOException

getTrainingDatasets

public Object getTrainingDatasets(@NonNull
                                  @NonNull String name)
                           throws FeatureStoreException,
                                  IOException

Description copied from class: FeatureStoreBase

Get all versions of a training dataset object from the selected feature store.

Specified by:

getTrainingDatasets in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Parameters:

name - name of the training dataset

Returns:

TrainingDataset

Throws:

FeatureStoreException

IOException

getOrCreateFeatureView

public FeatureView getOrCreateFeatureView(String name,
                                          com.logicalclocks.hsfs.flink.constructor.Query query,
                                          Integer version)
                                   throws FeatureStoreException,
                                          IOException

Specified by:

getOrCreateFeatureView in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

FeatureStoreException

IOException

getOrCreateFeatureView

public FeatureView getOrCreateFeatureView(String name,
                                          com.logicalclocks.hsfs.flink.constructor.Query query,
                                          Integer version,
                                          String description,
                                          List<String> labels)
                                   throws FeatureStoreException,
                                          IOException

Specified by:

getOrCreateFeatureView in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Throws:

FeatureStoreException

IOException

getFeatureView

public FeatureView getFeatureView(@NonNull
                                  @NonNull String name,
                                  @NonNull
                                  @NonNull Integer version)
                           throws FeatureStoreException,
                                  IOException

Get a feature view object from the selected feature store.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureView fv = fs.getFeatureView("fv_name", 1);
 
 

Specified by:

getFeatureView in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Parameters:

name - Name of the feature view.

version - Version to get.

Returns:

FeatureView The feature view metadata object.

Throws:

FeatureStoreException - If unable to retrieve FeatureView from the feature store.

IOException - Generic IO exception.

getFeatureView

public FeatureView getFeatureView(String name)
                           throws FeatureStoreException,
                                  IOException

Get a feature view object from the selected feature store.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureView fv = fs.getFeatureView("fv_name", 1);
 
 

Specified by:

getFeatureView in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Parameters:

name - Name of the feature view.

Returns:

FeatureView The feature view metadata object.

Throws:

FeatureStoreException - If unable to retrieve FeatureView from the feature store.

IOException - Generic IO exception.

getExternalFeatureGroup

public Object getExternalFeatureGroup(@NonNull
                                      @NonNull String name,
                                      @NonNull
                                      @NonNull Integer version)
                               throws FeatureStoreException,
                                      IOException

Description copied from class: FeatureStoreBase

Get a external feature group object from the feature store.

Specified by:

getExternalFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Parameters:

name - the name of the feature group

version - the version of the feature group

Returns:

ExternalFeatureGroup

Throws:

FeatureStoreException

IOException

getExternalFeatureGroup

public Object getExternalFeatureGroup(String name)
                               throws FeatureStoreException,
                                      IOException

Description copied from class: FeatureStoreBase

Get a external feature group object with default version `1` from the feature store.

Specified by:

getExternalFeatureGroup in class FeatureStoreBase<com.logicalclocks.hsfs.flink.constructor.Query>

Parameters:

name - the name of the feature group

Returns:

ExternalFeatureGroup

Throws:

FeatureStoreException

IOException