FeatureStore (hsfs-parent 3.9.0-RC14 API)

java.lang.Object
- com.logicalclocks.hsfs.FeatureStoreBase<Query>
- - com.logicalclocks.hsfs.spark.FeatureStore

public class FeatureStore
extends FeatureStoreBase<Query>

Field Summary
- Fields inherited from class com.logicalclocks.hsfs.FeatureStoreBase
  DEFAULT_VERSION, featureGroupApi, LOGGER, storageConnectorApi, trainingDatasetApi

Constructor Summary

Constructors
Constructor and Description

FeatureStore()

Constructors
Constructor and Description
`FeatureStore()`

Method Summary

All Methods Instance Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`com.logicalclocks.hsfs.spark.ExternalFeatureGroup.ExternalFeatureGroupBuilder`	`createExternalFeatureGroup()` Create external feature group builder object.
`com.logicalclocks.hsfs.spark.FeatureGroup.FeatureGroupBuilder`	`createFeatureGroup()` Create feature group builder object.
`FeatureView.FeatureViewBuilder`	`createFeatureView()`
`com.logicalclocks.hsfs.spark.ExternalFeatureGroup.ExternalFeatureGroupBuilder`	`createOnDemandFeatureGroup()` Deprecated.
`StreamFeatureGroup`	`createStreamFeatureGroup(@NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKey, List<String> partitionKey, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, StorageConnector storageConnector, String path)` Create a feature group object.
`com.logicalclocks.hsfs.spark.TrainingDataset.TrainingDatasetBuilder`	`createTrainingDataset()` Deprecated.
`ExternalFeatureGroup`	`getExternalFeatureGroup(String name)` Get an external feature group object with default version `1` from the feature store.
`ExternalFeatureGroup`	`getExternalFeatureGroup(@NonNull String name, @NonNull Integer version)` Get an external feature group object from the feature store.
`scala.collection.Seq<ExternalFeatureGroup>`	`getExternalFeatureGroups(@NonNull String name)` Get a list of all versions of an external feature group from the feature store.
`FeatureGroup`	`getFeatureGroup(String name)` Get a feature group object with default version `1` from the feature store.
`FeatureGroup`	`getFeatureGroup(@NonNull String name, @NonNull Integer version)` Get a feature group object from the feature store.
`scala.collection.Seq<FeatureGroup>`	`getFeatureGroups(@NonNull String name)` Get a list of all versions of a feature group from the feature store.
`FeatureView`	`getFeatureView(String name)` Get a feature view object with the default version `1` from the selected feature store.
`FeatureView`	`getFeatureView(@NonNull String name, @NonNull Integer version)` Get a feature view object from the selected feature store.
`ExternalFeatureGroup`	`getOnDemandFeatureGroup(String name)` Deprecated.
`ExternalFeatureGroup`	`getOnDemandFeatureGroup(@NonNull String name, @NonNull Integer version)` Deprecated.
`scala.collection.Seq<ExternalFeatureGroup>`	`getOnDemandFeatureGroups(@NonNull String name)` Deprecated.
`FeatureGroup`	`getOrCreateFeatureGroup(String name, Integer version)` Get feature group metadata object or create a new one if it doesn't exist.
`FeatureGroup`	`getOrCreateFeatureGroup(String name, Integer version, List<String> primaryKeys, boolean onlineEnabled, String eventTime)` Get feature group metadata object or create a new one if it doesn't exist.
`FeatureGroup`	`getOrCreateFeatureGroup(String name, Integer version, List<String> primaryKeys, List<String> partitionKeys, boolean onlineEnabled, String eventTime)` Get feature group metadata object or create a new one if it doesn't exist.
`FeatureGroup`	`getOrCreateFeatureGroup(String name, Integer version, String description, List<String> primaryKeys, List<String> partitionKeys, String hudiPrecombineKey, boolean onlineEnabled, TimeTravelFormat timeTravelFormat, StatisticsConfig statisticsConfig, String topicName, String notificationTopicName, String eventTime)` Get feature group metadata object or create a new one if it doesn't exist.
`FeatureView`	`getOrCreateFeatureView(String name, Query query, Integer version)` Get feature view metadata object or create a new one if it doesn't exist.
`FeatureView`	`getOrCreateFeatureView(String name, Query query, Integer version, String description, List<String> labels)` Get feature view metadata object or create a new one if it doesn't exist.
`StreamFeatureGroup`	`getOrCreateStreamFeatureGroup(String name, Integer version)` Get stream feature group metadata object or create a new one if it doesn't exist.
`StreamFeatureGroup`	`getOrCreateStreamFeatureGroup(String name, Integer version, List<String> primaryKeys, boolean onlineEnabled, String eventTime)` Get stream feature group metadata object or create a new one if it doesn't exist.
`StreamFeatureGroup`	`getOrCreateStreamFeatureGroup(String name, Integer version, List<String> primaryKeys, List<String> partitionKeys, boolean onlineEnabled, String eventTime)` Get stream feature group metadata object or create a new one if it doesn't exist.
`StreamFeatureGroup`	`getOrCreateStreamFeatureGroup(@NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, StorageConnector storageConnector, String path)` Get a feature group metadata object or create a new one if it doesn't exists.
`StreamFeatureGroup`	`getOrCreateStreamFeatureGroup(String name, Integer version, String description, List<String> primaryKeys, List<String> partitionKeys, String hudiPrecombineKey, boolean onlineEnabled, StatisticsConfig statisticsConfig, String eventTime, TimeTravelFormat timeTravelFormat)` Get stream feature group metadata object or create a new one if it doesn't exist.
`StreamFeatureGroup`	`getStreamFeatureGroup(String name)` Get a stream feature group object with default version `1` from the feature store.
`StreamFeatureGroup`	`getStreamFeatureGroup(@NonNull String name, @NonNull Integer version)` Get a stream feature group object from the feature store.
`TrainingDataset`	`getTrainingDataset(String name)` Deprecated.
`TrainingDataset`	`getTrainingDataset(@NonNull String name, @NonNull Integer version)` Deprecated.
`scala.collection.Seq<TrainingDataset>`	`getTrainingDatasets(@NonNull String name)` Deprecated.
`org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>`	`sql(String query)`

Methods inherited from class com.logicalclocks.hsfs.FeatureStoreBase
getAdlsConnector, getBigqueryConnector, getGcsConnector, getHopsFsConnector, getJdbcConnector, getKafkaConnector, getOnlineStorageConnector, getRedshiftConnector, getS3Connector, getSnowflakeConnector, getStorageConnector, toString

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail
- FeatureStore
```
public FeatureStore()
```

Method Detail

getFeatureGroup

public FeatureGroup getFeatureGroup(@NonNull
                                    @NonNull String name,
                                    @NonNull
                                    @NonNull Integer version)
                             throws FeatureStoreException,
                                    IOException

Get a feature group object from the feature store.

Feature group metadata handle enables to interact with the feature group, such as read the data or use the `Query`-API to perform joins between feature groups and create feature views.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getFeatureGroup("electricity_prices", 1);

Parameters:: name - the name of the feature group; version - the version of the feature group
Returns:: FeatureGroup The feature group metadata object.
Throws:: FeatureStoreException - If unable to retrieve feature group from the feature store.; IOException - Generic IO exception.

getFeatureGroup
```
public FeatureGroup getFeatureGroup(String name)
                             throws FeatureStoreException,
                                    IOException
```
Get a feature group object with default version `1` from the feature store.
Feature group metadata handle enables to interact with the feature group, such as read the data or use the `Query`-API to perform joins between feature groups and create feature views.
```
 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getFeatureGroup("electricity_prices");
 
 
```
Parameters:

name - the name of the feature group

Returns:

FeatureGroup The feature group metadata object.

Throws:

FeatureStoreException - If unable to retrieve feature group from the feature store.

IOException - Generic IO exception.

getFeatureGroups

public scala.collection.Seq<FeatureGroup> getFeatureGroups(@NonNull
                                                           @NonNull String name)
                                                    throws FeatureStoreException,
                                                           IOException

Get a list of all versions of a feature group from the feature store.

Feature group metadata handle enables to interact with the feature group, such as read the data or use the `Query`-API to perform joins between feature groups and create feature views.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getFeatureGroups("electricity_prices");

Parameters:: name - the name of the feature group
Returns:: List of FeatureGroup metadata objects.
Throws:: FeatureStoreException - If unable to retrieve feature group from the feature store.; IOException - Generic IO exception.

createFeatureGroup

public com.logicalclocks.hsfs.spark.FeatureGroup.FeatureGroupBuilder createFeatureGroup()

Create feature group builder object.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup.FeatureGroupBuilder = fs.createFeatureGroup()

Returns:: FeatureGroup.FeatureGroupBuilder a FeatureGroup builder object.

getOrCreateFeatureGroup

public FeatureGroup getOrCreateFeatureGroup(String name,
                                            Integer version)
                                     throws IOException,
                                            FeatureStoreException

Get feature group metadata object or create a new one if it doesn't exist. This method doesn't update existing feature group metadata.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getOrCreateFeatureGroup("fg_name", 1);

Parameters:: name - of the feature group to retrieve or create.; version - of the feature group to retrieve or create.
Returns:: FeatureGroup The feature group metadata object.
Throws:: IOException - Generic IO exception.; FeatureStoreException - If unable to retrieve FeatureGroup from the feature store.

getOrCreateFeatureGroup
```
public FeatureGroup getOrCreateFeatureGroup(String name,
                                            Integer version,
                                            List<String> primaryKeys,
                                            boolean onlineEnabled,
                                            String eventTime)
                                     throws IOException,
                                            FeatureStoreException
```
Get feature group metadata object or create a new one if it doesn't exist. This method doesn't update existing feature group metadata.
```
 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getOrCreateFeatureGroup("fg_name", 1, primaryKeys, true, "datetime");
 
 
```
Parameters:

name - of the feature group to retrieve or create.

version - of the feature group to retrieve or create.

primaryKeys - A list of feature names to be used as primary key for the feature group. This primary key can be a composite key of multiple features and will be used as joining key.

onlineEnabled - Define whether the feature group should be made available also in the online feature store for low latency access.

eventTime - Name of the feature containing the event time for the features in this feature group. If eventTime is set the feature group can be used for point-in-time joins. The supported data types for the eventTime column are: timestamp, date and bigint

Returns:

FeatureGroup: The feature group metadata object

Throws:

IOException - Generic IO exception.

FeatureStoreException - If unable to retrieve FeatureGroup from the feature store.

getOrCreateFeatureGroup
```
public FeatureGroup getOrCreateFeatureGroup(String name,
                                            Integer version,
                                            List<String> primaryKeys,
                                            List<String> partitionKeys,
                                            boolean onlineEnabled,
                                            String eventTime)
                                     throws IOException,
                                            FeatureStoreException
```
Get feature group metadata object or create a new one if it doesn't exist. This method doesn't update existing feature group metadata.
```
 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getOrCreateFeatureGroup("fg_name", 1, primaryKeys, partitionKeys, true, "datetime");
 
 
```
Parameters:

name - of the feature group to retrieve or create.

version - of the feature group to retrieve or create.

primaryKeys - A list of feature names to be used as primary key for the feature group. This primary key can be a composite key of multiple features and will be used as joining key.

partitionKeys - A list of feature names to be used as partition key when writing the feature data to the offline storage.

onlineEnabled - Define whether the feature group should be made available also in the online feature store for low latency access.

eventTime - Name of the feature containing the event time for the features in this feature group. If eventTime is set the feature group can be used for point-in-time joins. The supported data types for the eventTime column are: timestamp, date and bigint

Returns:

FeatureGroup: The feature group metadata object

Throws:

IOException - Generic IO exception.

FeatureStoreException - If unable to retrieve FeatureGroup from the feature store.

getOrCreateFeatureGroup
```
public FeatureGroup getOrCreateFeatureGroup(String name,
                                            Integer version,
                                            String description,
                                            List<String> primaryKeys,
                                            List<String> partitionKeys,
                                            String hudiPrecombineKey,
                                            boolean onlineEnabled,
                                            TimeTravelFormat timeTravelFormat,
                                            StatisticsConfig statisticsConfig,
                                            String topicName,
                                            String notificationTopicName,
                                            String eventTime)
                                     throws IOException,
                                            FeatureStoreException
```
Get feature group metadata object or create a new one if it doesn't exist. This method doesn't update existing feature group metadata.
```
 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getOrCreateFeatureGroup("fg_name", 1, primaryKeys, partitionKeys, true, "datetime");
 
 
```
Parameters:

name - of the feature group to retrieve or create.

version - of the feature group to retrieve or create.

description - contents of the feature group to improve discoverability for Data Scientists

primaryKeys - A list of feature names to be used as primary key for the feature group. This primary key can be a composite key of multiple features and will be used as joining key.

partitionKeys - A list of feature names to be used as partition key when writing the feature data to the offline storage.

hudiPrecombineKey - A feature name to be used as a precombine key for the `HUDI` feature group. If feature group has time travel format `HUDI` and hudi precombine key was not specified then the first primary key of the feature group will be used as hudi precombine key.

onlineEnabled - Define whether the feature group should be made available also in the online feature store for low latency access.

timeTravelFormat - Format used for time travel: `TimeTravelFormat.HUDI` or `TimeTravelFormat.NONE`.

statisticsConfig - A configuration object, to generally enable descriptive statistics computation for this feature group, `"correlations`" to turn on feature correlation computation, `"histograms"` to compute feature value frequencies and `"exact_uniqueness"` to compute uniqueness, distinctness and entropy. The values should be booleans indicating the setting. To fully turn off statistics computation pass `statisticsConfig=false`.

topicName - Optionally, define the name of the topic used for data ingestion. If left undefined it defaults to using project topic.

notificationTopicName - Optionally, define the name of the topic used for sending notifications when entries are inserted or updated on the online feature store. If left undefined no notifications are sent.

eventTime - Name of the feature containing the event time for the features in this feature group. If eventTime is set the feature group can be used for point-in-time joins. The supported data types for the eventTime column are: timestamp, date and bigint

Returns:

FeatureGroup: The feature group metadata object.

Throws:

IOException - Generic IO exception.

FeatureStoreException - If unable to retrieve FeatureGroup from the feature store.

getStreamFeatureGroup
```
public StreamFeatureGroup getStreamFeatureGroup(String name)
                                         throws FeatureStoreException,
                                                IOException
```
Get a stream feature group object with default version `1` from the feature store.
Getting a stream feature group metadata handle enables to interact with the feature group, such as read the data or use the `Query`-API to perform joins between feature groups and create feature views.
```
 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getStreamFeatureGroup("electricity_prices");
 <
 
```
Specified by:

getStreamFeatureGroup in class FeatureStoreBase<Query>

Parameters:

name - the name of the feature group

Returns:

FeatureGroup

Throws:

FeatureStoreException - If unable to retrieve feature group from the feature store.

IOException - Generic IO exception.

getStreamFeatureGroup

public StreamFeatureGroup getStreamFeatureGroup(@NonNull
                                                @NonNull String name,
                                                @NonNull
                                                @NonNull Integer version)
                                         throws FeatureStoreException,
                                                IOException

Get a stream feature group object from the feature store.

Getting a stream feature group metadata handle enables to interact with the feature group, such as read the data or use the `Query`-API to perform joins between feature groups and create feature views.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getStreamFeatureGroup("electricity_prices", 1);

Specified by:: getStreamFeatureGroup in class FeatureStoreBase<Query>
Parameters:: name - the name of the feature group; version - the version of the feature group
Returns:: StreamFeatureGroup The stream feature group metadata object.
Throws:: FeatureStoreException - If unable to retrieve feature group from the feature store.; IOException - Generic IO exception.

createStreamFeatureGroup

public StreamFeatureGroup createStreamFeatureGroup(@NonNull
                                                   @NonNull String name,
                                                   Integer version,
                                                   String description,
                                                   Boolean onlineEnabled,
                                                   TimeTravelFormat timeTravelFormat,
                                                   List<String> primaryKey,
                                                   List<String> partitionKey,
                                                   String eventTime,
                                                   String hudiPrecombineKey,
                                                   List<Feature> features,
                                                   StatisticsConfig statisticsConfig,
                                                   StorageConnector storageConnector,
                                                   String path)

Description copied from class: FeatureStoreBase

Create a feature group object.

Specified by:: createStreamFeatureGroup in class FeatureStoreBase<Query>
Parameters:: name - the name of the feature group; version - the version of the feature group; description - descrption of the feature group; onlineEnabled - whether the feature group should be online enabled; timeTravelFormat - the data format to use to store the offline data; primaryKey - list of primary keys; partitionKey - list of partition keys; eventTime - the feature/column to use as event time; hudiPrecombineKey - if the timeTravelFormat is set to hudi, the feature/column to use as precombine key; features - the list of feature objects if defined explicitly; statisticsConfig - the statistics configuration for the feature group; storageConnector - the storage connector to use to store the offline feature data (Default stored internally in Hopsworks).; path - the path on the storage where to store the feature data.
Returns:: The feature group metadata object.

getOrCreateStreamFeatureGroup

public StreamFeatureGroup getOrCreateStreamFeatureGroup(@NonNull
                                                        @NonNull String name,
                                                        Integer version,
                                                        String description,
                                                        Boolean onlineEnabled,
                                                        TimeTravelFormat timeTravelFormat,
                                                        List<String> primaryKeys,
                                                        List<String> partitionKeys,
                                                        String eventTime,
                                                        String hudiPrecombineKey,
                                                        List<Feature> features,
                                                        StatisticsConfig statisticsConfig,
                                                        StorageConnector storageConnector,
                                                        String path)
                                                 throws IOException,
                                                        FeatureStoreException

Description copied from class: FeatureStoreBase

Get a feature group metadata object or create a new one if it doesn't exists.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        StreamFeatureGroup streamFeatureGroup = featureStore.getOrCreateStreamFeatureGroup(
                 "doc_example",
                 1,
                 "Feature group for example in documentation",
                 true,
                 TimeTravelFormat.HUDI,
                 Collections.singletonList("primary_key"),
                 Collections.singletonList("partition_key"),
                 "event_time",
                 null,
                 features,
                 null,
                 new StatisticsConfig(true, true, true, true),
                 null
         );

         streamFeatureGroup.save()

Specified by:: getOrCreateStreamFeatureGroup in class FeatureStoreBase<Query>
Parameters:: name - the name of the feature group; version - the version of the feature group; description - descrption of the feature group; onlineEnabled - whether the feature group should be online enabled; timeTravelFormat - the data format to use to store the offline data; primaryKeys - list of primary keys; partitionKeys - list of partition keys; eventTime - the feature/column to use as event time; hudiPrecombineKey - if the timeTravelFormat is set to hudi, the feature/column to use as precombine key; features - the list of feature objects if defined explicitly; statisticsConfig - the statistics configuration for the feature group; storageConnector - the storage connector to use to store the offline feature data (Default stored internally in Hopsworks).; path - the path on the storage where to store the feature data.
Returns:: The feature group metadata object.
Throws:: IOException; FeatureStoreException

getOrCreateStreamFeatureGroup

public StreamFeatureGroup getOrCreateStreamFeatureGroup(String name,
                                                        Integer version)
                                                 throws IOException,
                                                        FeatureStoreException

Get stream feature group metadata object or create a new one if it doesn't exist. This method doesn't update existing feature group metadata.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getOrCreateStreamFeatureGroup("fg_name", 1);

Parameters:: name - of the feature group to retrieve or create.; version - of the feature group to retrieve or create.
Returns:: FeatureGroup The feature group metadata object.
Throws:: IOException - Generic IO exception.; FeatureStoreException - If unable to retrieve StreamFeatureGroup from the feature store.

getOrCreateStreamFeatureGroup
```
public StreamFeatureGroup getOrCreateStreamFeatureGroup(String name,
                                                        Integer version,
                                                        List<String> primaryKeys,
                                                        boolean onlineEnabled,
                                                        String eventTime)
                                                 throws IOException,
                                                        FeatureStoreException
```
Get stream feature group metadata object or create a new one if it doesn't exist. This method doesn't update existing feature group metadata.
```
 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getOrCreateStreamFeatureGroup("fg_name", 1, primaryKeys, true, "datetime");
 
 
```
Parameters:

name - Name of the feature group to retrieve or create.

version - Version of the feature group to retrieve or create.

primaryKeys - A list of feature names to be used as primary key for the feature group. This primary key can be a composite key of multiple features and will be used as joining key.

onlineEnabled - Define whether the feature group should be made available also in the online feature store for low latency access.

eventTime - Name of the feature containing the event time for the features in this feature group. If eventTime is set the feature group can be used for point-in-time joins. The supported data types for the eventTime column are: timestamp, date and bigint

Returns:

FeatureGroup The feature group metadata object.

Throws:

IOException - Generic IO exception.

FeatureStoreException - If unable to retrieve StreamFeatureGroup from the feature store.

getOrCreateStreamFeatureGroup
```
public StreamFeatureGroup getOrCreateStreamFeatureGroup(String name,
                                                        Integer version,
                                                        List<String> primaryKeys,
                                                        List<String> partitionKeys,
                                                        boolean onlineEnabled,
                                                        String eventTime)
                                                 throws IOException,
                                                        FeatureStoreException
```
Get stream feature group metadata object or create a new one if it doesn't exist. This method doesn't update existing feature group metadata.
```
 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getOrCreateStreamFeatureGroup("fg_name", 1, primaryKeys, partitionKeys, true,
        "datetime");
 
 
```
Parameters:

name - Name of the feature group to retrieve or create.

version - Version of the feature group to retrieve or create.

primaryKeys - A list of feature names to be used as primary key for the feature group. This primary key can be a composite key of multiple features and will be used as joining key.

partitionKeys - A list of feature names to be used as partition key when writing the feature data to the offline storage.

onlineEnabled - Define whether the feature group should be made available also in the online feature store for low latency access.

eventTime - Name of the feature containing the event time for the features in this feature group. If eventTime is set the feature group can be used for point-in-time joins. The supported data types for the eventTime column are: timestamp, date and bigint

Returns:

FeatureGroup: The feature group metadata object

Throws:

IOException - Generic IO exception.

FeatureStoreException - If unable to retrieve StreamFeatureGroup from the feature store.

getOrCreateStreamFeatureGroup
```
public StreamFeatureGroup getOrCreateStreamFeatureGroup(String name,
                                                        Integer version,
                                                        String description,
                                                        List<String> primaryKeys,
                                                        List<String> partitionKeys,
                                                        String hudiPrecombineKey,
                                                        boolean onlineEnabled,
                                                        StatisticsConfig statisticsConfig,
                                                        String eventTime,
                                                        TimeTravelFormat timeTravelFormat)
                                                 throws IOException,
                                                        FeatureStoreException
```
Get stream feature group metadata object or create a new one if it doesn't exist. This method doesn't update existing feature group metadata.
```
 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        StreamFeatureGroup fg = fs.getOrCreateStreamFeatureGroup("fg_name", 1, primaryKeys,
        partitionKeys, true, "datetime");
 
 
```
Parameters:

name - of the feature group to retrieve or create.

version - of the feature group to retrieve or create.

description - contents of the feature group to improve discoverability for Data Scientists

primaryKeys - A list of feature names to be used as primary key for the feature group. This primary key can be a composite key of multiple features and will be used as joining key.

partitionKeys - A list of feature names to be used as partition key when writing the feature data to the offline storage.

hudiPrecombineKey - A feature name to be used as a precombine key for the `HUDI` feature group. If feature group has time travel format `HUDI` and hudi precombine key was not specified then the first primary key of the feature group will be used as hudi precombine key.

onlineEnabled - Define whether the feature group should be made available also in the online feature store for low latency access.

timeTravelFormat - Format used for time travel, defaults to `"HUDI"`.

statisticsConfig - A configuration object, to generally enable descriptive statistics computation for this feature group, `"correlations`" to turn on feature correlation computation, `"histograms"` to compute feature value frequencies and `"exact_uniqueness"` to compute uniqueness, distinctness and entropy. The values should be booleans indicating the setting. To fully turn off statistics computation pass `statisticsConfig=false`.

eventTime - Name of the feature containing the event time for the features in this feature group. If eventTime is set the feature group can be used for point-in-time joins.

Returns:

FeatureGroup: The feature group metadata object.

Throws:

IOException - Generic IO exception.

FeatureStoreException - If unable to retrieve FeatureGroup from the feature store.

createExternalFeatureGroup

public com.logicalclocks.hsfs.spark.ExternalFeatureGroup.ExternalFeatureGroupBuilder createExternalFeatureGroup()

Create external feature group builder object.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        ExternalFeatureGroup.ExternalFeatureGroupBuilder = fs.createExternalFeatureGroup()

Returns:: ExternalFeatureGroup.ExternalFeatureGroupBuilder a ExternalFeatureGroup builder object.

createOnDemandFeatureGroup

@Deprecated
public com.logicalclocks.hsfs.spark.ExternalFeatureGroup.ExternalFeatureGroupBuilder createOnDemandFeatureGroup()

Deprecated.

getExternalFeatureGroups

public scala.collection.Seq<ExternalFeatureGroup> getExternalFeatureGroups(@NonNull
                                                                           @NonNull String name)
                                                                    throws FeatureStoreException,
                                                                           IOException

Get a list of all versions of an external feature group from the feature store.

Feature group metadata handle enables to interact with the feature group, such as read the data or use the `Query`-API to perform joins between feature groups and create feature views.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureGroup fg = fs.getExternalFeatureGroups("external_fg_name");

Parameters:: name - The name of the feature group.
Returns:: List of ExternalFeatureGroup metadata objects.
Throws:: FeatureStoreException - If unable to retrieve feature group from the feature store.; IOException - Generic IO exception.

getExternalFeatureGroup

public ExternalFeatureGroup getExternalFeatureGroup(@NonNull
                                                    @NonNull String name,
                                                    @NonNull
                                                    @NonNull Integer version)
                                             throws FeatureStoreException,
                                                    IOException

Get an external feature group object from the feature store.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        ExternalFeatureGroup fg = fs.getExternalFeatureGroup("external_fg_name", 1);

Parameters:: name - the name of the feature group; version - the version of the feature group
Returns:: ExternalFeatureGroup The external feature group metadata object.
Throws:: FeatureStoreException - If unable to retrieve feature group from the feature store.; IOException - Generic IO exception.

getExternalFeatureGroup
```
public ExternalFeatureGroup getExternalFeatureGroup(String name)
                                             throws FeatureStoreException,
                                                    IOException
```
Get an external feature group object with default version `1` from the feature store.
Getting external feature group metadata handle enables to interact with the feature group, such as read the data or use the `Query`-API to perform joins between feature groups and create feature views.
```
 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        ExternalFeatureGroup fg = fs.getExternalFeatureGroup("external_fg_name");
 
 
```
Parameters:

name - the name of the feature group

Returns:

ExternalFeatureGroup The external feature group metadata object.

Throws:

FeatureStoreException - If unable to retrieve feature group from the feature store.

IOException - Generic IO exception.

getOnDemandFeatureGroup

@Deprecated
public ExternalFeatureGroup getOnDemandFeatureGroup(@NonNull
                                                                @NonNull String name,
                                                                @NonNull
                                                                @NonNull Integer version)
                                                         throws FeatureStoreException,
                                                                IOException

Deprecated.

Throws:: FeatureStoreException; IOException

getOnDemandFeatureGroup

@Deprecated
public ExternalFeatureGroup getOnDemandFeatureGroup(String name)
                                                         throws FeatureStoreException,
                                                                IOException

Deprecated.

Throws:: FeatureStoreException; IOException

getOnDemandFeatureGroups

@Deprecated
public scala.collection.Seq<ExternalFeatureGroup> getOnDemandFeatureGroups(@NonNull
                                                                                       @NonNull String name)
                                                                                throws FeatureStoreException,
                                                                                       IOException

Deprecated.

Throws:: FeatureStoreException; IOException

createFeatureView

public FeatureView.FeatureViewBuilder createFeatureView()

getOrCreateFeatureView

public FeatureView getOrCreateFeatureView(String name,
                                          Query query,
                                          Integer version)
                                   throws FeatureStoreException,
                                          IOException

Get feature view metadata object or create a new one if it doesn't exist. This method doesn't update existing feature view metadata.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureView fv = fs.getOrCreateFeatureView("fv_name", query, 1);

Parameters:: name - Name of the feature view.; query - Query object.; version - Version of the feature view.
Returns:: FeatureView The feature view metadata object.
Throws:: FeatureStoreException - If unable to retrieve FeatureView from the feature store.; IOException - Generic IO exception.

getOrCreateFeatureView

public FeatureView getOrCreateFeatureView(String name,
                                          Query query,
                                          Integer version,
                                          String description,
                                          List<String> labels)
                                   throws FeatureStoreException,
                                          IOException

Get feature view metadata object or create a new one if it doesn't exist. This method doesn't update existing feature view metadata.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureView fv = fs.getOrCreateFeatureView("fv_name", query, 1, description, labels);
        "datetime");

Parameters:: name - Name of the feature view.; query - Query object.; version - Version of the feature view.; description - Description of the feature view.; labels - List of label features.
Returns:: FeatureView The feature view metadata object.
Throws:: FeatureStoreException - If unable to retrieve FeatureView from the feature store.; IOException - Generic IO exception.

getFeatureView

public FeatureView getFeatureView(@NonNull
                                  @NonNull String name,
                                  @NonNull
                                  @NonNull Integer version)
                           throws FeatureStoreException,
                                  IOException

Get a feature view object from the selected feature store.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureView fv = fs.getFeatureView("fv_name", 1);

Specified by:: getFeatureView in class FeatureStoreBase<Query>
Parameters:: name - Name of the feature view.; version - Version to get.
Returns:: FeatureView The feature view metadata object.
Throws:: FeatureStoreException - If unable to retrieve FeatureView from the feature store.; IOException - Generic IO exception.

getFeatureView

public FeatureView getFeatureView(String name)
                           throws FeatureStoreException,
                                  IOException

Get a feature view object with the default version `1` from the selected feature store.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        FeatureView fv = fs.getFeatureView("fv_name");

Specified by:: getFeatureView in class FeatureStoreBase<Query>
Parameters:: name - Name of the feature view.
Returns:: FeatureView The feature view metadata object.
Throws:: FeatureStoreException - If unable to retrieve FeatureView from the feature store.; IOException - Generic IO exception.

sql

public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> sql(String query)

createTrainingDataset

@Deprecated
public com.logicalclocks.hsfs.spark.TrainingDataset.TrainingDatasetBuilder createTrainingDataset()

Deprecated.

getTrainingDataset

@Deprecated
public TrainingDataset getTrainingDataset(@NonNull
                                                      @NonNull String name,
                                                      @NonNull
                                                      @NonNull Integer version)
                                               throws FeatureStoreException,
                                                      IOException

Deprecated.

Throws:: FeatureStoreException; IOException

getTrainingDataset

@Deprecated
public TrainingDataset getTrainingDataset(String name)
                                               throws FeatureStoreException,
                                                      IOException

Deprecated.

Throws:: FeatureStoreException; IOException

getTrainingDatasets

@Deprecated
public scala.collection.Seq<TrainingDataset> getTrainingDatasets(@NonNull
                                                                             @NonNull String name)
                                                                      throws FeatureStoreException,
                                                                             IOException

Deprecated.

Throws:: FeatureStoreException; IOException

Class FeatureStore

Field Summary

Fields inherited from class com.logicalclocks.hsfs.FeatureStoreBase

Constructor Summary

Method Summary

Methods inherited from class com.logicalclocks.hsfs.FeatureStoreBase

Methods inherited from class java.lang.Object

Constructor Detail

FeatureStore

Method Detail

getFeatureGroup

getFeatureGroup

getFeatureGroups

createFeatureGroup

getOrCreateFeatureGroup

getOrCreateFeatureGroup

getOrCreateFeatureGroup

getOrCreateFeatureGroup

getStreamFeatureGroup

getStreamFeatureGroup

createStreamFeatureGroup

getOrCreateStreamFeatureGroup

getOrCreateStreamFeatureGroup

getOrCreateStreamFeatureGroup

getOrCreateStreamFeatureGroup

getOrCreateStreamFeatureGroup

createExternalFeatureGroup

createOnDemandFeatureGroup

getExternalFeatureGroups

getExternalFeatureGroup

getExternalFeatureGroup

getOnDemandFeatureGroup

getOnDemandFeatureGroup

getOnDemandFeatureGroups

createFeatureView

getOrCreateFeatureView

getOrCreateFeatureView

getFeatureView

getFeatureView

sql

createTrainingDataset

getTrainingDataset

getTrainingDataset

getTrainingDatasets