Package | Description |
---|---|
com.logicalclocks.hsfs.spark | |
com.logicalclocks.hsfs.spark.engine |
Modifier and Type | Method and Description |
---|---|
FeatureGroup |
FeatureStore.getFeatureGroup(String name)
Get a feature group object with default version `1` from the feature store.
|
FeatureGroup |
FeatureStore.getFeatureGroup(@NonNull String name,
@NonNull Integer version)
Get a feature group object from the feature store.
|
FeatureGroup |
FeatureStore.getOrCreateFeatureGroup(String name,
Integer version)
Get feature group metadata object or create a new one if it doesn't exist.
|
FeatureGroup |
FeatureStore.getOrCreateFeatureGroup(String name,
Integer version,
List<String> primaryKeys,
boolean onlineEnabled,
String eventTime)
Get feature group metadata object or create a new one if it doesn't exist.
|
FeatureGroup |
FeatureStore.getOrCreateFeatureGroup(String name,
Integer version,
List<String> primaryKeys,
List<String> partitionKeys,
boolean onlineEnabled,
String eventTime)
Get feature group metadata object or create a new one if it doesn't exist.
|
FeatureGroup |
FeatureStore.getOrCreateFeatureGroup(String name,
Integer version,
String description,
List<String> primaryKeys,
List<String> partitionKeys,
String hudiPrecombineKey,
boolean onlineEnabled,
TimeTravelFormat timeTravelFormat,
StatisticsConfig statisticsConfig,
String eventTime)
Get feature group metadata object or create a new one if it doesn't exist.
|
Modifier and Type | Method and Description |
---|---|
scala.collection.Seq<FeatureGroup> |
FeatureStore.getFeatureGroups(@NonNull String name)
Get a list of all versions of a feature group from the feature store.
|
Modifier and Type | Method and Description |
---|---|
FeatureGroup |
FeatureGroupEngine.getFeatureGroup(FeatureStore featureStore,
String fgName,
Integer fgVersion) |
FeatureGroup |
FeatureGroupEngine.getOrCreateFeatureGroup(FeatureStore featureStore,
String name,
Integer version,
String description,
List<String> primaryKeys,
List<String> partitionKeys,
String hudiPrecombineKey,
boolean onlineEnabled,
TimeTravelFormat timeTravelFormat,
StatisticsConfig statisticsConfig,
String eventTime) |
FeatureGroup |
FeatureGroupEngine.save(FeatureGroup featureGroup,
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataset,
List<String> partitionKeys,
String hudiPrecombineKey,
Map<String,String> writeOptions)
Create the metadata and write the data to the online/offline feature store.
|
FeatureGroup |
FeatureGroupEngine.saveFeatureGroupMetaData(FeatureGroup featureGroup,
List<String> partitionKeys,
String hudiPrecombineKey,
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> featureData,
boolean saveEmpty) |
Modifier and Type | Method and Description |
---|---|
List<FeatureGroup> |
FeatureGroupEngine.getFeatureGroups(FeatureStore featureStore,
String fgName) |
Modifier and Type | Method and Description |
---|---|
void |
FeatureGroupEngine.insert(FeatureGroup featureGroup,
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> featureData,
Storage storage,
HudiOperationType operation,
org.apache.spark.sql.SaveMode saveMode,
List<String> partitionKeys,
String hudiPrecombineKey,
Map<String,String> writeOptions) |
org.apache.spark.sql.streaming.StreamingQuery |
FeatureGroupEngine.insertStream(FeatureGroup featureGroup,
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> featureData,
String queryName,
String outputMode,
boolean awaitTermination,
Long timeout,
String checkpointLocation,
List<String> partitionKeys,
String hudiPrecombineKey,
Map<String,String> writeOptions)
Deprecated.
|
FeatureGroup |
FeatureGroupEngine.save(FeatureGroup featureGroup,
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataset,
List<String> partitionKeys,
String hudiPrecombineKey,
Map<String,String> writeOptions)
Create the metadata and write the data to the online/offline feature store.
|
void |
FeatureGroupEngine.saveDataframe(FeatureGroup featureGroup,
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataset,
Storage storage,
HudiOperationType operation,
Map<String,String> offlineWriteOptions,
Map<String,String> onlineWriteOptions,
Integer validationId) |
FeatureGroup |
FeatureGroupEngine.saveFeatureGroupMetaData(FeatureGroup featureGroup,
List<String> partitionKeys,
String hudiPrecombineKey,
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> featureData,
boolean saveEmpty) |
Copyright © 2023. All rights reserved.