com.logicalclocks.hsfs

Class StreamFeatureGroup<T>

java.lang.Object

com.logicalclocks.hsfs.FeatureGroupBase<List<T>>

com.logicalclocks.hsfs.StreamFeatureGroup<T>

public class StreamFeatureGroup<T>
extends FeatureGroupBase<List<T>>

Field Summary

Fields

Modifier and Type	Field and Description
protected FeatureGroupEngine	featureGroupEngine

Fields inherited from class com.logicalclocks.hsfs.FeatureGroupBase

created, creator, deltaStreamerJobConf, deprecated, description, eventTime, expectationsNames, featureGroupEngineBase, features, featureStore, hudiPrecombineKey, id, location, LOGGER, name, notificationTopicName, onlineEnabled, onlineTopicName, partitionKeys, path, primaryKeys, statisticColumns, statisticsConfig, storageConnector, subject, timeTravelFormat, topicName, type, utils, version

Constructor Summary

Constructors

Constructor and Description
StreamFeatureGroup()
StreamFeatureGroup(FeatureStoreBase featureStore, int id)
StreamFeatureGroup(FeatureStoreBase featureStore, @NonNull String name, Integer version, String description, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String hudiPrecombineKey, boolean onlineEnabled, List<Feature> features, StatisticsConfig statisticsConfig, String onlineTopicName, String eventTime, StorageConnector storageConnector, String path)
StreamFeatureGroup(Integer id, String description, List<Feature> features)

Method Summary

Modifier and Type	Method and Description
List<T>	insertStream(List<T> featureData)
List<T>	insertStream(List<T> featureData, Map<String,String> writeOptions)
void	save() Save the feature group metadata on Hopsworks.
void	save(Map<String,String> writeOptions, JobConfiguration materializationJobConfiguration) Save the feature group metadata on Hopsworks.

Methods inherited from class com.logicalclocks.hsfs.FeatureGroupBase

addTag, checkDeprecated, delete, deleteTag, getAvroSchema, getComplexFeatures, getDeserializedAvroSchema, getDeserializedEncodedAvroSchema, getEncodedAvroSchema, getFeature, getFeatureAvroSchema, getPrimaryKeys, getSubject, getTag, getTags, setDeprecated, unloadSubject, updateDeprecated, updateDeprecated, updateDescription, updateFeatureDescription, updateNotificationTopicName, updateStatisticsConfig

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

featureGroupEngine

protected FeatureGroupEngine featureGroupEngine

Constructor Detail

StreamFeatureGroup

public StreamFeatureGroup(FeatureStoreBase featureStore,
                          @NonNull
                          @NonNull String name,
                          Integer version,
                          String description,
                          TimeTravelFormat timeTravelFormat,
                          List<String> primaryKeys,
                          List<String> partitionKeys,
                          String hudiPrecombineKey,
                          boolean onlineEnabled,
                          List<Feature> features,
                          StatisticsConfig statisticsConfig,
                          String onlineTopicName,
                          String eventTime,
                          StorageConnector storageConnector,
                          String path)

StreamFeatureGroup

public StreamFeatureGroup()

StreamFeatureGroup

public StreamFeatureGroup(Integer id,
                          String description,
                          List<Feature> features)

StreamFeatureGroup

public StreamFeatureGroup(FeatureStoreBase featureStore,
                          int id)

Method Detail

save

public void save()
          throws FeatureStoreException,
                 IOException

Save the feature group metadata on Hopsworks. This method is idempotent, if the feature group already exists the method does nothing.

Throws:

FeatureStoreException

IOException

save

public void save(Map<String,String> writeOptions,
                 JobConfiguration materializationJobConfiguration)
          throws FeatureStoreException,
                 IOException

Save the feature group metadata on Hopsworks. This method is idempotent, if the feature group already exists, the method does nothing

Parameters:

writeOptions - Options to provide to the materialization job

materializationJobConfiguration - Resource configuration for the materialization job

Throws:

FeatureStoreException

IOException

insertStream

public List<T> insertStream(List<T> featureData)
                     throws Exception

Specified by:

insertStream in class FeatureGroupBase<List<T>>

Throws:

Exception

insertStream

public List<T> insertStream(List<T> featureData,
                            Map<String,String> writeOptions)
                     throws Exception

Specified by:

insertStream in class FeatureGroupBase<List<T>>

Throws:

Exception