com.logicalclocks.hsfs

Class FeatureGroupBaseForApi

java.lang.Object

com.logicalclocks.hsfs.FeatureGroupBase<FeatureGroupBaseForApi>

com.logicalclocks.hsfs.FeatureGroupBaseForApi

public class FeatureGroupBaseForApi
extends FeatureGroupBase<FeatureGroupBaseForApi>

Field Summary

Fields inherited from class com.logicalclocks.hsfs.FeatureGroupBase

created, creator, deltaStreamerJobConf, description, eventTime, expectationsNames, featureGroupEngineBase, features, featureStore, hudiPrecombineKey, id, location, LOGGER, name, onlineEnabled, onlineTopicName, partitionKeys, primaryKeys, statisticColumns, statisticsConfig, subject, timeTravelFormat, type, utils, version

Constructor Summary

Constructors

Constructor and Description
FeatureGroupBaseForApi(FeatureStoreBase featureStore, Integer id)

Method Summary

Modifier and Type	Method and Description
void	appendFeatures(Feature features) Append a single feature to the schema of the feature group.
void	appendFeatures(List<Feature> features) Append features to the schema of the feature group.
QueryBase	asOf(String wallclockTime)
QueryBase	asOf(String wallclockTime, String excludeUntil)
void	commitDeleteRecord(FeatureGroupBaseForApi featureData)
void	commitDeleteRecord(FeatureGroupBaseForApi featureData, Map<String,String> writeOptions)
Map<Long,Map<String,String>>	commitDetails()
Map<Long,Map<String,String>>	commitDetails(Integer limit)
Map<Long,Map<String,String>>	commitDetails(String wallclockTime)
Map<Long,Map<String,String>>	commitDetails(String wallclockTime, Integer limit)
Statistics	computeStatistics() Recompute the statistics for the feature group and save them to the feature store.
Statistics	computeStatistics(String wallclockTime)
Statistics	getStatistics() Get the last statistics commit for the feature group.
void	insert(FeatureGroupBaseForApi featureData)
void	insert(FeatureGroupBaseForApi featureData, boolean overwrite)
void	insert(FeatureGroupBaseForApi featureData, boolean overwrite, Map<String,String> writeOptions)
void	insert(FeatureGroupBaseForApi featureData, boolean overwrite, Map<String,String> writeOptions, JobConfiguration jobConfiguration)
void	insert(FeatureGroupBaseForApi featureData, HudiOperationType operation)
void	insert(FeatureGroupBaseForApi featureData, JobConfiguration jobConfiguration)
void	insert(FeatureGroupBaseForApi featureData, Map<String,String> writeOptions)
void	insert(FeatureGroupBaseForApi featureData, Storage storage)
void	insert(FeatureGroupBaseForApi featureData, Storage storage, boolean overwrite)
void	insert(FeatureGroupBaseForApi featureData, Storage storage, boolean overwrite, HudiOperationType operation, Map<String,String> writeOptions)
Object	insertStream(FeatureGroupBaseForApi featureData)
Object	insertStream(FeatureGroupBaseForApi featureData, Map<String,String> writeOptions)
Object	insertStream(FeatureGroupBaseForApi featureData, String queryName)
Object	insertStream(FeatureGroupBaseForApi featureData, String queryName, Map<String,String> writeOptions)
Object	insertStream(FeatureGroupBaseForApi featureData, String queryName, String outputMode)
Object	insertStream(FeatureGroupBaseForApi featureData, String queryName, String outputMode, boolean awaitTermination, Long timeout)
Object	insertStream(FeatureGroupBaseForApi featureData, String queryName, String outputMode, boolean awaitTermination, Long timeout, String checkpointLocation)
Object	insertStream(FeatureGroupBaseForApi featureData, String queryName, String outputMode, boolean awaitTermination, Long timeout, String checkpointLocation, Map<String,String> writeOptions)
Object	insertStream(FeatureGroupBaseForApi featureData, String queryName, String outputMode, boolean awaitTermination, Long timeout, String checkpointLocation, Map<String,String> writeOptions, JobConfiguration jobConfiguration)
Object	insertStream(FeatureGroupBaseForApi featureData, String queryName, String outputMode, boolean awaitTermination, String checkpointLocation)
Object	insertStream(FeatureGroupBaseForApi featureData, String queryName, String outputMode, String checkpointLocation)
FeatureGroupBaseForApi	read()
FeatureGroupBaseForApi	read(boolean online)
FeatureGroupBaseForApi	read(boolean online, Map<String,String> readOptions)
FeatureGroupBaseForApi	read(Map<String,String> readOptions)
FeatureGroupBaseForApi	read(String wallclockTime)
FeatureGroupBaseForApi	read(String wallclockTime, Map<String,String> readOptions)
QueryBase	select(List<String> features)
QueryBase	selectAll()
QueryBase	selectExcept(List<String> features)
QueryBase	selectExceptFeatures(List<Feature> features)
QueryBase	selectFeatures(List<Feature> features)
void	show(int numRows)
void	show(int numRows, boolean online)
void	updateFeatures(Feature feature) Update the metadata of multiple features.
void	updateFeatures(List<Feature> features) Update the metadata of multiple features.

Methods inherited from class com.logicalclocks.hsfs.FeatureGroupBase

addTag, delete, deleteTag, getAvroSchema, getComplexFeatures, getDeserializedAvroSchema, getDeserializedEncodedAvroSchema, getEncodedAvroSchema, getFeature, getFeatureAvroSchema, getPrimaryKeys, getSubject, getTag, getTags, unloadSubject, updateDescription, updateFeatureDescription, updateStatisticsConfig

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FeatureGroupBaseForApi

public FeatureGroupBaseForApi(FeatureStoreBase featureStore,
                              Integer id)

Method Detail

read

public FeatureGroupBaseForApi read()
                            throws FeatureStoreException,
                                   IOException

Specified by:

read in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

read

public FeatureGroupBaseForApi read(boolean online)
                            throws FeatureStoreException,
                                   IOException

Specified by:

read in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

read

public FeatureGroupBaseForApi read(Map<String,String> readOptions)
                            throws FeatureStoreException,
                                   IOException

Specified by:

read in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

read

public FeatureGroupBaseForApi read(boolean online,
                                   Map<String,String> readOptions)
                            throws FeatureStoreException,
                                   IOException

Specified by:

read in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

read

public FeatureGroupBaseForApi read(String wallclockTime)
                            throws FeatureStoreException,
                                   IOException,
                                   ParseException

Specified by:

read in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

ParseException

read

public FeatureGroupBaseForApi read(String wallclockTime,
                                   Map<String,String> readOptions)
                            throws FeatureStoreException,
                                   IOException,
                                   ParseException

Specified by:

read in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

ParseException

asOf

public QueryBase asOf(String wallclockTime)
               throws FeatureStoreException,
                      ParseException

Specified by:

asOf in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

ParseException

asOf

public QueryBase asOf(String wallclockTime,
                      String excludeUntil)
               throws FeatureStoreException,
                      ParseException

Specified by:

asOf in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

ParseException

show

public void show(int numRows)
          throws FeatureStoreException,
                 IOException

Specified by:

show in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

show

public void show(int numRows,
                 boolean online)
          throws FeatureStoreException,
                 IOException

Specified by:

show in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

insert

public void insert(FeatureGroupBaseForApi featureData)
            throws IOException,
                   FeatureStoreException,
                   ParseException

Specified by:

insert in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

IOException

FeatureStoreException

ParseException

insert

public void insert(FeatureGroupBaseForApi featureData,
                   Map<String,String> writeOptions)
            throws FeatureStoreException,
                   IOException,
                   ParseException

Specified by:

insert in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

ParseException

insert

public void insert(FeatureGroupBaseForApi featureData,
                   Storage storage)
            throws IOException,
                   FeatureStoreException,
                   ParseException

Specified by:

insert in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

IOException

FeatureStoreException

ParseException

insert

public void insert(FeatureGroupBaseForApi featureData,
                   boolean overwrite)
            throws IOException,
                   FeatureStoreException,
                   ParseException

Specified by:

insert in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

IOException

FeatureStoreException

ParseException

insert

public void insert(FeatureGroupBaseForApi featureData,
                   Storage storage,
                   boolean overwrite)
            throws IOException,
                   FeatureStoreException,
                   ParseException

Specified by:

insert in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

IOException

FeatureStoreException

ParseException

insert

public void insert(FeatureGroupBaseForApi featureData,
                   boolean overwrite,
                   Map<String,String> writeOptions)
            throws FeatureStoreException,
                   IOException,
                   ParseException

Specified by:

insert in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

ParseException

insert

public void insert(FeatureGroupBaseForApi featureData,
                   HudiOperationType operation)
            throws FeatureStoreException,
                   IOException,
                   ParseException

Specified by:

insert in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

ParseException

insert

public void insert(FeatureGroupBaseForApi featureData,
                   Storage storage,
                   boolean overwrite,
                   HudiOperationType operation,
                   Map<String,String> writeOptions)
            throws FeatureStoreException,
                   IOException,
                   ParseException

Specified by:

insert in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

ParseException

insert

public void insert(FeatureGroupBaseForApi featureData,
                   JobConfiguration jobConfiguration)
            throws FeatureStoreException,
                   IOException,
                   ParseException

Specified by:

insert in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

ParseException

insert

public void insert(FeatureGroupBaseForApi featureData,
                   boolean overwrite,
                   Map<String,String> writeOptions,
                   JobConfiguration jobConfiguration)
            throws FeatureStoreException,
                   IOException,
                   ParseException

Specified by:

insert in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

ParseException

commitDeleteRecord

public void commitDeleteRecord(FeatureGroupBaseForApi featureData)
                        throws FeatureStoreException,
                               IOException,
                               ParseException

Specified by:

commitDeleteRecord in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

ParseException

commitDeleteRecord

public void commitDeleteRecord(FeatureGroupBaseForApi featureData,
                               Map<String,String> writeOptions)
                        throws FeatureStoreException,
                               IOException,
                               ParseException

Specified by:

commitDeleteRecord in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

ParseException

commitDetails

public Map<Long,Map<String,String>> commitDetails()
                                           throws IOException,
                                                  FeatureStoreException,
                                                  ParseException

Specified by:

commitDetails in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

IOException

FeatureStoreException

ParseException

commitDetails

public Map<Long,Map<String,String>> commitDetails(Integer limit)
                                           throws IOException,
                                                  FeatureStoreException,
                                                  ParseException

Specified by:

commitDetails in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

IOException

FeatureStoreException

ParseException

commitDetails

public Map<Long,Map<String,String>> commitDetails(String wallclockTime)
                                           throws IOException,
                                                  FeatureStoreException,
                                                  ParseException

Specified by:

commitDetails in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

IOException

FeatureStoreException

ParseException

commitDetails

public Map<Long,Map<String,String>> commitDetails(String wallclockTime,
                                                  Integer limit)
                                           throws IOException,
                                                  FeatureStoreException,
                                                  ParseException

Specified by:

commitDetails in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

IOException

FeatureStoreException

ParseException

selectFeatures

public QueryBase selectFeatures(List<Feature> features)

Specified by:

selectFeatures in class FeatureGroupBase<FeatureGroupBaseForApi>

select

public QueryBase select(List<String> features)

Specified by:

select in class FeatureGroupBase<FeatureGroupBaseForApi>

selectAll

public QueryBase selectAll()

Specified by:

selectAll in class FeatureGroupBase<FeatureGroupBaseForApi>

selectExceptFeatures

public QueryBase selectExceptFeatures(List<Feature> features)

Specified by:

selectExceptFeatures in class FeatureGroupBase<FeatureGroupBaseForApi>

selectExcept

public QueryBase selectExcept(List<String> features)

Specified by:

selectExcept in class FeatureGroupBase<FeatureGroupBaseForApi>

insertStream

public Object insertStream(FeatureGroupBaseForApi featureData)
                    throws Exception

Specified by:

insertStream in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

Exception

insertStream

public Object insertStream(FeatureGroupBaseForApi featureData,
                           String queryName)
                    throws Exception

Specified by:

insertStream in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

Exception

insertStream

public Object insertStream(FeatureGroupBaseForApi featureData,
                           Map<String,String> writeOptions)
                    throws Exception

Specified by:

insertStream in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

Exception

insertStream

public Object insertStream(FeatureGroupBaseForApi featureData,
                           String queryName,
                           Map<String,String> writeOptions)
                    throws Exception

Specified by:

insertStream in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

Exception

insertStream

public Object insertStream(FeatureGroupBaseForApi featureData,
                           String queryName,
                           String outputMode)
                    throws Exception

Specified by:

insertStream in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

Exception

insertStream

public Object insertStream(FeatureGroupBaseForApi featureData,
                           String queryName,
                           String outputMode,
                           String checkpointLocation)
                    throws Exception

Specified by:

insertStream in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

Exception

insertStream

public Object insertStream(FeatureGroupBaseForApi featureData,
                           String queryName,
                           String outputMode,
                           boolean awaitTermination,
                           Long timeout)
                    throws Exception

Specified by:

insertStream in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

Exception

insertStream

public Object insertStream(FeatureGroupBaseForApi featureData,
                           String queryName,
                           String outputMode,
                           boolean awaitTermination,
                           Long timeout,
                           String checkpointLocation)
                    throws Exception

Specified by:

insertStream in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

Exception

insertStream

public Object insertStream(FeatureGroupBaseForApi featureData,
                           String queryName,
                           String outputMode,
                           boolean awaitTermination,
                           Long timeout,
                           String checkpointLocation,
                           Map<String,String> writeOptions)
                    throws Exception

Specified by:

insertStream in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

Exception

insertStream

public Object insertStream(FeatureGroupBaseForApi featureData,
                           String queryName,
                           String outputMode,
                           boolean awaitTermination,
                           String checkpointLocation)
                    throws Exception

Specified by:

insertStream in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

Exception

insertStream

public Object insertStream(FeatureGroupBaseForApi featureData,
                           String queryName,
                           String outputMode,
                           boolean awaitTermination,
                           Long timeout,
                           String checkpointLocation,
                           Map<String,String> writeOptions,
                           JobConfiguration jobConfiguration)
                    throws Exception

Specified by:

insertStream in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

Exception

updateFeatures

public void updateFeatures(List<Feature> features)
                    throws FeatureStoreException,
                           IOException,
                           ParseException

Description copied from class: FeatureGroupBase

Update the metadata of multiple features. Currently only feature description updates are supported.

Specified by:

updateFeatures in class FeatureGroupBase<FeatureGroupBaseForApi>

Parameters:

features - List of Feature metadata objects

Throws:

FeatureStoreException - FeatureStoreException

IOException - IOException

ParseException - ParseException

updateFeatures

public void updateFeatures(Feature feature)
                    throws FeatureStoreException,
                           IOException,
                           ParseException

Description copied from class: FeatureGroupBase

Update the metadata of multiple features. Currently only feature description updates are supported.

Specified by:

updateFeatures in class FeatureGroupBase<FeatureGroupBaseForApi>

Parameters:

feature - Feature metadata object

Throws:

FeatureStoreException - FeatureStoreException

IOException - IOException

ParseException - ParseException

appendFeatures

public void appendFeatures(List<Feature> features)
                    throws FeatureStoreException,
                           IOException,
                           ParseException

Description copied from class: FeatureGroupBase

Append features to the schema of the feature group. It is only possible to append features to a feature group. Removing features is considered a breaking change.

Specified by:

appendFeatures in class FeatureGroupBase<FeatureGroupBaseForApi>

Parameters:

features - list of Feature metadata objects

Throws:

FeatureStoreException - FeatureStoreException

IOException - IOException

ParseException - ParseException

appendFeatures

public void appendFeatures(Feature features)
                    throws FeatureStoreException,
                           IOException,
                           ParseException

Description copied from class: FeatureGroupBase

Append a single feature to the schema of the feature group. It is only possible to append features to a feature group. Removing features is considered a breaking change.

Specified by:

appendFeatures in class FeatureGroupBase<FeatureGroupBaseForApi>

Parameters:

features - List of Feature metadata objects

Throws:

FeatureStoreException - FeatureStoreException

IOException - IOException

ParseException - ParseException

computeStatistics

public Statistics computeStatistics()
                             throws FeatureStoreException,
                                    IOException,
                                    ParseException

Description copied from class: FeatureGroupBase

Recompute the statistics for the feature group and save them to the feature store.

Specified by:

computeStatistics in class FeatureGroupBase<FeatureGroupBaseForApi>

Returns:

statistics object of computed statistics

Throws:

FeatureStoreException - FeatureStoreException

IOException - IOException

ParseException

computeStatistics

public Statistics computeStatistics(String wallclockTime)
                             throws FeatureStoreException,
                                    IOException,
                                    ParseException

Specified by:

computeStatistics in class FeatureGroupBase<FeatureGroupBaseForApi>

Throws:

FeatureStoreException

IOException

ParseException

getStatistics

public Statistics getStatistics()
                         throws FeatureStoreException,
                                IOException

Description copied from class: FeatureGroupBase

Get the last statistics commit for the feature group.

Specified by:

getStatistics in class FeatureGroupBase<FeatureGroupBaseForApi>

Returns:

statistics object of latest commit

Throws:

FeatureStoreException - FeatureStoreException

IOException - IOException