Package com.logicalclocks.hsfs

Class StreamFeatureGroup<T>

java.lang.Object

com.logicalclocks.hsfs.FeatureGroupBase<List<T>>

com.logicalclocks.hsfs.StreamFeatureGroup<T>

public class StreamFeatureGroup<T>
extends FeatureGroupBase<List<T>>

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	StreamFeatureGroup.StreamFeatureGroupBuilder<T>

Field Summary

Fields

Modifier and Type	Field	Description
protected FeatureGroupEngine	featureGroupEngine

Fields inherited from class com.logicalclocks.hsfs.FeatureGroupBase

created, creator, dataSource, deltaStreamerJobConf, deprecated, description, eventTime, expectationsNames, featureGroupEngineBase, features, featureStore, featurestoreId, hudiPrecombineKey, id, location, LOGGER, name, notificationTopicName, onlineConfig, onlineEnabled, onlineIngestionApi, onlineTopicName, partitionKeys, primaryKeys, statisticColumns, statisticsConfig, subject, timeTravelFormat, topicName, type, utils, version

Constructor Summary

Constructors

Constructor	Description
StreamFeatureGroup()
StreamFeatureGroup(FeatureStoreBase featureStore, int id)
StreamFeatureGroup(FeatureStoreBase featureStore, @NonNull String name, Integer version, String description, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String hudiPrecombineKey, boolean onlineEnabled, List<Feature> features, StatisticsConfig statisticsConfig, String onlineTopicName, String eventTime, StorageConnector storageConnector, String path, OnlineConfig onlineConfig)
StreamFeatureGroup(Integer id, String description, List<Feature> features)

Method Summary

Modifier and Type	Method	Description
List<T>	insertStream(List<T> featureData)
List<T>	insertStream(List<T> featureData, Map<String,String> writeOptions)
void	save()	Save the feature group metadata on Hopsworks.
void	save(Map<String,String> writeOptions, JobConfiguration materializationJobConfiguration)	Save the feature group metadata on Hopsworks.
Query	select(List<String> features)	Select a subset of features of the feature group and return a query object.
Query	selectAll()	Select all features of the feature group and return a query object.
Query	selectExcept(List<String> features)	Select all features including primary key and event time feature of the feature group except provided `features` and return a query object.
Query	selectExceptFeatures(List<Feature> features)	Select all features including primary key and event time feature of the feature group except provided `features` and return a query object.
Query	selectFeatures(List<Feature> features)	Select a subset of features of the feature group and return a query object.

Methods inherited from class com.logicalclocks.hsfs.FeatureGroupBase

addTag, checkDeprecated, delete, deleteTag, getAvroSchema, getComplexFeatures, getDeserializedAvroSchema, getDeserializedEncodedAvroSchema, getEncodedAvroSchema, getFeature, getFeatureAvroSchema, getLatestOnlineIngestion, getOnlineIngestion, getPrimaryKeys, getSubject, getTag, getTags, setDataSource, setDeprecated, unloadSubject, updateDeprecated, updateDeprecated, updateDescription, updateFeatureDescription, updateNotificationTopicName, updateStatisticsConfig

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

featureGroupEngine

protected FeatureGroupEngine featureGroupEngine

Constructor Details

StreamFeatureGroup

public StreamFeatureGroup(FeatureStoreBase featureStore,
 @NonNull
 @NonNull String name,
 Integer version,
 String description,
 TimeTravelFormat timeTravelFormat,
 List<String> primaryKeys,
 List<String> partitionKeys,
 String hudiPrecombineKey,
 boolean onlineEnabled,
 List<Feature> features,
 StatisticsConfig statisticsConfig,
 String onlineTopicName,
 String eventTime,
 StorageConnector storageConnector,
 String path,
 OnlineConfig onlineConfig)

StreamFeatureGroup

public StreamFeatureGroup()

StreamFeatureGroup

public StreamFeatureGroup(Integer id,
 String description,
 List<Feature> features)

StreamFeatureGroup

public StreamFeatureGroup(FeatureStoreBase featureStore,
 int id)

Method Details

save

public void save()
          throws FeatureStoreException,
IOException

Save the feature group metadata on Hopsworks. This method is idempotent, if the feature group already exists the method does nothing.

Throws:

FeatureStoreException

IOException

save

public void save(Map<String,String> writeOptions,
 JobConfiguration materializationJobConfiguration)
          throws FeatureStoreException,
IOException

Save the feature group metadata on Hopsworks. This method is idempotent, if the feature group already exists, the method does nothing

Parameters:

writeOptions - Options to provide to the materialization job

materializationJobConfiguration - Resource configuration for the materialization job

Throws:

FeatureStoreException

IOException

insertStream

public List<T> insertStream(List<T> featureData)
                     throws Exception

Specified by:

insertStream in class FeatureGroupBase<List<T>>

Throws:

Exception

insertStream

public List<T> insertStream(List<T> featureData,
 Map<String,String> writeOptions)
                     throws Exception

Specified by:

insertStream in class FeatureGroupBase<List<T>>

Throws:

Exception

selectFeatures

public Query selectFeatures(List<Feature> features)

Select a subset of features of the feature group and return a query object. The query can be used to construct joins of feature groups or create a feature view with a subset of features of the feature group.

Parameters:

features - List of Feature meta data objects.

Returns:

Query object.

select

public Query select(List<String> features)

Select a subset of features of the feature group and return a query object. The query can be used to construct joins of feature groups or create a feature view with a subset of features of the feature group.

Parameters:

features - List of Feature names.

Returns:

Query object.

selectAll

public Query selectAll()

Select all features of the feature group and return a query object. The query can be used to construct joins of feature groups or create a feature view with a subset of features of the feature group.

Returns:

Query object.

selectExceptFeatures

public Query selectExceptFeatures(List<Feature> features)

Select all features including primary key and event time feature of the feature group except provided `features` and return a query object. The query can be used to construct joins of feature groups or create a feature view with a subset of features of the feature group.

Parameters:

features - List of Feature meta data objects.

Returns:

Query object.

selectExcept

public Query selectExcept(List<String> features)

Select all features including primary key and event time feature of the feature group except provided `features` and return a query object. The query can be used to construct joins of feature groups or create a feature view with a subset of features of the feature group.

Parameters:

features - List of Feature names.

Returns:

Query object.