com.logicalclocks.hsfs

Class StreamFeatureGroup<T>

java.lang.Object

com.logicalclocks.hsfs.FeatureGroupBase<List<T>>

com.logicalclocks.hsfs.StreamFeatureGroup<T>

public class StreamFeatureGroup<T>
extends FeatureGroupBase<List<T>>

Field Summary

Fields

Modifier and Type	Field and Description
protected FeatureGroupEngine	featureGroupEngine

Fields inherited from class com.logicalclocks.hsfs.FeatureGroupBase

created, creator, deltaStreamerJobConf, deprecated, description, eventTime, expectationsNames, featureGroupEngineBase, features, featureStore, featurestoreId, hudiPrecombineKey, id, location, LOGGER, name, notificationTopicName, onlineEnabled, onlineTopicName, partitionKeys, path, primaryKeys, statisticColumns, statisticsConfig, storageConnector, subject, timeTravelFormat, topicName, type, utils, version

Constructor Summary

Constructors

Constructor and Description
StreamFeatureGroup()
StreamFeatureGroup(FeatureStoreBase featureStore, int id)
StreamFeatureGroup(FeatureStoreBase featureStore, @NonNull String name, Integer version, String description, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String hudiPrecombineKey, boolean onlineEnabled, List<Feature> features, StatisticsConfig statisticsConfig, String onlineTopicName, String eventTime, StorageConnector storageConnector, String path)
StreamFeatureGroup(Integer id, String description, List<Feature> features)

Method Summary

Modifier and Type	Method and Description
List<T>	insertStream(List<T> featureData)
List<T>	insertStream(List<T> featureData, Map<String,String> writeOptions)
void	save() Save the feature group metadata on Hopsworks.
void	save(Map<String,String> writeOptions, JobConfiguration materializationJobConfiguration) Save the feature group metadata on Hopsworks.
Query	select(List<String> features) Select a subset of features of the feature group and return a query object.
Query	selectAll() Select all features of the feature group and return a query object.
Query	selectExcept(List<String> features) Select all features including primary key and event time feature of the feature group except provided `features` and return a query object.
Query	selectExceptFeatures(List<Feature> features) Select all features including primary key and event time feature of the feature group except provided `features` and return a query object.
Query	selectFeatures(List<Feature> features) Select a subset of features of the feature group and return a query object.

Methods inherited from class com.logicalclocks.hsfs.FeatureGroupBase

addTag, checkDeprecated, delete, deleteTag, getAvroSchema, getComplexFeatures, getDeserializedAvroSchema, getDeserializedEncodedAvroSchema, getEncodedAvroSchema, getFeature, getFeatureAvroSchema, getPrimaryKeys, getSubject, getTag, getTags, setDeprecated, unloadSubject, updateDeprecated, updateDeprecated, updateDescription, updateFeatureDescription, updateNotificationTopicName, updateStatisticsConfig

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

featureGroupEngine

protected FeatureGroupEngine featureGroupEngine

Constructor Detail

StreamFeatureGroup

public StreamFeatureGroup(FeatureStoreBase featureStore,
                          @NonNull
                          @NonNull String name,
                          Integer version,
                          String description,
                          TimeTravelFormat timeTravelFormat,
                          List<String> primaryKeys,
                          List<String> partitionKeys,
                          String hudiPrecombineKey,
                          boolean onlineEnabled,
                          List<Feature> features,
                          StatisticsConfig statisticsConfig,
                          String onlineTopicName,
                          String eventTime,
                          StorageConnector storageConnector,
                          String path)

StreamFeatureGroup

public StreamFeatureGroup()

StreamFeatureGroup

public StreamFeatureGroup(Integer id,
                          String description,
                          List<Feature> features)

StreamFeatureGroup

public StreamFeatureGroup(FeatureStoreBase featureStore,
                          int id)

Method Detail

save

public void save()
          throws FeatureStoreException,
                 IOException

Save the feature group metadata on Hopsworks. This method is idempotent, if the feature group already exists the method does nothing.

Throws:

FeatureStoreException

IOException

save

public void save(Map<String,String> writeOptions,
                 JobConfiguration materializationJobConfiguration)
          throws FeatureStoreException,
                 IOException

Save the feature group metadata on Hopsworks. This method is idempotent, if the feature group already exists, the method does nothing

Parameters:

writeOptions - Options to provide to the materialization job

materializationJobConfiguration - Resource configuration for the materialization job

Throws:

FeatureStoreException

IOException

insertStream

public List<T> insertStream(List<T> featureData)
                     throws Exception

Specified by:

insertStream in class FeatureGroupBase<List<T>>

Throws:

Exception

insertStream

public List<T> insertStream(List<T> featureData,
                            Map<String,String> writeOptions)
                     throws Exception

Specified by:

insertStream in class FeatureGroupBase<List<T>>

Throws:

Exception

selectFeatures

public Query selectFeatures(List<Feature> features)

Select a subset of features of the feature group and return a query object. The query can be used to construct joins of feature groups or create a feature view with a subset of features of the feature group.

Parameters:

features - List of Feature meta data objects.

Returns:

Query object.

select

public Query select(List<String> features)

Select a subset of features of the feature group and return a query object. The query can be used to construct joins of feature groups or create a feature view with a subset of features of the feature group.

Parameters:

features - List of Feature names.

Returns:

Query object.

selectAll

public Query selectAll()

Select all features of the feature group and return a query object. The query can be used to construct joins of feature groups or create a feature view with a subset of features of the feature group.

Returns:

Query object.

selectExceptFeatures

public Query selectExceptFeatures(List<Feature> features)

Select all features including primary key and event time feature of the feature group except provided `features` and return a query object. The query can be used to construct joins of feature groups or create a feature view with a subset of features of the feature group.

Parameters:

features - List of Feature meta data objects.

Returns:

Query object.

selectExcept

public Query selectExcept(List<String> features)

Select all features including primary key and event time feature of the feature group except provided `features` and return a query object. The query can be used to construct joins of feature groups or create a feature view with a subset of features of the feature group.

Parameters:

features - List of Feature names.

Returns:

Query object.