| Package | Description | 
|---|---|
| com.logicalclocks.hsfs.spark | |
| com.logicalclocks.hsfs.spark.engine | 
| Modifier and Type | Method and Description | 
|---|---|
FeatureGroup | 
FeatureStore.getFeatureGroup(String name)
Get a feature group object with default version `1` from the feature store. 
 | 
FeatureGroup | 
FeatureStore.getFeatureGroup(@NonNull String name,
               @NonNull Integer version)
Get a feature group object from the feature store. 
 | 
FeatureGroup | 
FeatureStore.getOrCreateFeatureGroup(String name,
                       Integer version)
Get feature group metadata object or create a new one if it doesn't exist. 
 | 
FeatureGroup | 
FeatureStore.getOrCreateFeatureGroup(String name,
                       Integer version,
                       List<String> primaryKeys,
                       boolean onlineEnabled,
                       String eventTime)
Get feature group metadata object or create a new one if it doesn't exist. 
 | 
FeatureGroup | 
FeatureStore.getOrCreateFeatureGroup(String name,
                       Integer version,
                       List<String> primaryKeys,
                       List<String> partitionKeys,
                       boolean onlineEnabled,
                       String eventTime)
Get feature group metadata object or create a new one if it doesn't exist. 
 | 
FeatureGroup | 
FeatureStore.getOrCreateFeatureGroup(String name,
                       Integer version,
                       String description,
                       List<String> primaryKeys,
                       List<String> partitionKeys,
                       String hudiPrecombineKey,
                       boolean onlineEnabled,
                       TimeTravelFormat timeTravelFormat,
                       StatisticsConfig statisticsConfig,
                       String eventTime)
Get feature group metadata object or create a new one if it doesn't exist. 
 | 
| Modifier and Type | Method and Description | 
|---|---|
scala.collection.Seq<FeatureGroup> | 
FeatureStore.getFeatureGroups(@NonNull String name)
Get a list of all versions of a feature group from the feature store. 
 | 
| Modifier and Type | Method and Description | 
|---|---|
FeatureGroup | 
FeatureGroupEngine.getFeatureGroup(FeatureStore featureStore,
               String fgName,
               Integer fgVersion)  | 
FeatureGroup | 
FeatureGroupEngine.getOrCreateFeatureGroup(FeatureStore featureStore,
                       String name,
                       Integer version,
                       String description,
                       List<String> primaryKeys,
                       List<String> partitionKeys,
                       String hudiPrecombineKey,
                       boolean onlineEnabled,
                       TimeTravelFormat timeTravelFormat,
                       StatisticsConfig statisticsConfig,
                       String eventTime)  | 
FeatureGroup | 
FeatureGroupEngine.save(FeatureGroup featureGroup,
    org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataset,
    List<String> partitionKeys,
    String hudiPrecombineKey,
    Map<String,String> writeOptions)
Create the metadata and write the data to the online/offline feature store. 
 | 
FeatureGroup | 
FeatureGroupEngine.saveFeatureGroupMetaData(FeatureGroup featureGroup,
                        List<String> partitionKeys,
                        String hudiPrecombineKey,
                        org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> featureData,
                        boolean saveEmpty)  | 
| Modifier and Type | Method and Description | 
|---|---|
List<FeatureGroup> | 
FeatureGroupEngine.getFeatureGroups(FeatureStore featureStore,
                String fgName)  | 
| Modifier and Type | Method and Description | 
|---|---|
void | 
FeatureGroupEngine.insert(FeatureGroup featureGroup,
      org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> featureData,
      Storage storage,
      HudiOperationType operation,
      org.apache.spark.sql.SaveMode saveMode,
      List<String> partitionKeys,
      String hudiPrecombineKey,
      Map<String,String> writeOptions)  | 
org.apache.spark.sql.streaming.StreamingQuery | 
FeatureGroupEngine.insertStream(FeatureGroup featureGroup,
            org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> featureData,
            String queryName,
            String outputMode,
            boolean awaitTermination,
            Long timeout,
            String checkpointLocation,
            List<String> partitionKeys,
            String hudiPrecombineKey,
            Map<String,String> writeOptions)
Deprecated.  
 | 
FeatureGroup | 
FeatureGroupEngine.save(FeatureGroup featureGroup,
    org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataset,
    List<String> partitionKeys,
    String hudiPrecombineKey,
    Map<String,String> writeOptions)
Create the metadata and write the data to the online/offline feature store. 
 | 
void | 
FeatureGroupEngine.saveDataframe(FeatureGroup featureGroup,
             org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataset,
             Storage storage,
             HudiOperationType operation,
             Map<String,String> offlineWriteOptions,
             Map<String,String> onlineWriteOptions,
             Integer validationId)  | 
FeatureGroup | 
FeatureGroupEngine.saveFeatureGroupMetaData(FeatureGroup featureGroup,
                        List<String> partitionKeys,
                        String hudiPrecombineKey,
                        org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> featureData,
                        boolean saveEmpty)  | 
Copyright © 2023. All rights reserved.