| Package | Description | 
|---|---|
| com.logicalclocks.hsfs.spark | |
| com.logicalclocks.hsfs.spark.engine | |
| com.logicalclocks.hsfs.spark.engine.hudi | 
| Modifier and Type | Method and Description | 
|---|---|
StreamFeatureGroup | 
FeatureStore.getOrCreateStreamFeatureGroup(String name,
                             Integer version)
Get stream feature group metadata object or create a new one if it doesn't exist. 
 | 
StreamFeatureGroup | 
FeatureStore.getOrCreateStreamFeatureGroup(String name,
                             Integer version,
                             List<String> primaryKeys,
                             boolean onlineEnabled,
                             String eventTime)
Get stream feature group metadata object or create a new one if it doesn't exist. 
 | 
StreamFeatureGroup | 
FeatureStore.getOrCreateStreamFeatureGroup(String name,
                             Integer version,
                             List<String> primaryKeys,
                             List<String> partitionKeys,
                             boolean onlineEnabled,
                             String eventTime)
Get stream feature group metadata object or create a new one if it doesn't exist. 
 | 
StreamFeatureGroup | 
FeatureStore.getOrCreateStreamFeatureGroup(String name,
                             Integer version,
                             String description,
                             List<String> primaryKeys,
                             List<String> partitionKeys,
                             String hudiPrecombineKey,
                             boolean onlineEnabled,
                             StatisticsConfig statisticsConfig,
                             String eventTime)
Get stream feature group metadata object or create a new one if it doesn't exist. 
 | 
StreamFeatureGroup | 
FeatureStore.getStreamFeatureGroup(String name)
Get a stream feature group object with default version `1` from the feature store. 
 | 
StreamFeatureGroup | 
FeatureStore.getStreamFeatureGroup(@NonNull String name,
                     @NonNull Integer version)
Get a stream feature group object from the feature store. 
 | 
| Modifier and Type | Method and Description | 
|---|---|
StreamFeatureGroup | 
FeatureGroupEngine.getOrCreateStreamFeatureGroup(FeatureStore featureStore,
                             String name,
                             Integer version,
                             String description,
                             List<String> primaryKeys,
                             List<String> partitionKeys,
                             String hudiPrecombineKey,
                             boolean onlineEnabled,
                             StatisticsConfig statisticsConfig,
                             String eventTime)  | 
StreamFeatureGroup | 
FeatureGroupEngine.getStreamFeatureGroup(FeatureStore featureStore,
                     String fgName,
                     Integer fgVersion)  | 
StreamFeatureGroup | 
FeatureGroupEngine.save(StreamFeatureGroup featureGroup,
    org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataset,
    List<String> partitionKeys,
    String hudiPrecombineKey,
    Map<String,String> writeOptions,
    JobConfiguration sparkJobConfiguration)
Create the metadata and write the data to the online/offline feature store. 
 | 
StreamFeatureGroup | 
FeatureGroupEngine.saveFeatureGroupMetaData(StreamFeatureGroup featureGroup,
                        List<String> partitionKeys,
                        String hudiPrecombineKey,
                        Map<String,String> writeOptions,
                        JobConfiguration sparkJobConfiguration,
                        org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> featureData)  | 
| Modifier and Type | Method and Description | 
|---|---|
List<StreamFeatureGroup> | 
FeatureGroupEngine.getStreamFeatureGroups(FeatureStore featureStore,
                      String fgName)  | 
| Modifier and Type | Method and Description | 
|---|---|
void | 
FeatureGroupEngine.insert(StreamFeatureGroup streamFeatureGroup,
      org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> featureData,
      org.apache.spark.sql.SaveMode saveMode,
      List<String> partitionKeys,
      String hudiPrecombineKey,
      Map<String,String> writeOptions,
      JobConfiguration jobConfiguration)  | 
org.apache.spark.sql.streaming.StreamingQuery | 
FeatureGroupEngine.insertStream(StreamFeatureGroup streamFeatureGroup,
            org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> featureData,
            String queryName,
            String outputMode,
            boolean awaitTermination,
            Long timeout,
            String checkpointLocation,
            List<String> partitionKeys,
            String hudiPrecombineKey,
            Map<String,String> writeOptions,
            JobConfiguration jobConfiguration)  | 
StreamFeatureGroup | 
FeatureGroupEngine.save(StreamFeatureGroup featureGroup,
    org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataset,
    List<String> partitionKeys,
    String hudiPrecombineKey,
    Map<String,String> writeOptions,
    JobConfiguration sparkJobConfiguration)
Create the metadata and write the data to the online/offline feature store. 
 | 
StreamFeatureGroup | 
FeatureGroupEngine.saveFeatureGroupMetaData(StreamFeatureGroup featureGroup,
                        List<String> partitionKeys,
                        String hudiPrecombineKey,
                        Map<String,String> writeOptions,
                        JobConfiguration sparkJobConfiguration,
                        org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> featureData)  | 
void | 
SparkEngine.streamToHudiTable(StreamFeatureGroup streamFeatureGroup,
                 Map<String,String> writeOptions)  | 
| Modifier and Type | Method and Description | 
|---|---|
void | 
HudiEngine.streamToHoodieTable(org.apache.spark.sql.SparkSession sparkSession,
                   StreamFeatureGroup streamFeatureGroup,
                   Map<String,String> writeOptions)  | 
Copyright © 2023. All rights reserved.