Uses of Class
com.logicalclocks.hsfs.DataSource

Packages that use DataSource

Package	Description
com.logicalclocks.hsfs
com.logicalclocks.hsfs.beam
com.logicalclocks.hsfs.beam.engine
com.logicalclocks.hsfs.engine
com.logicalclocks.hsfs.flink
com.logicalclocks.hsfs.flink.engine
com.logicalclocks.hsfs.spark
com.logicalclocks.hsfs.spark.engine
com.logicalclocks.hsfs.spark.util

Uses of DataSource in com.logicalclocks.hsfs

Fields in com.logicalclocks.hsfs declared as DataSource

Modifier and Type	Field	Description
protected DataSource	FeatureGroupBase.dataSource
protected DataSource	TrainingDatasetBase.dataSource

Methods in com.logicalclocks.hsfs that return DataSource

Modifier and Type	Method	Description
DataSource	FeatureStoreBase.getDataSource(String name)	Get a previously created data source from the feature store.

Methods in com.logicalclocks.hsfs with parameters of type DataSource

Modifier and Type	Method	Description
StreamFeatureGroup	FeatureStore.createStreamFeatureGroup(@NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, DataSource dataSource)
abstract FeatureGroupBase	FeatureStoreBase.createStreamFeatureGroup(@NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, DataSource dataSource)	Create a stream feature group object.
Integer	FeatureViewBase.createTrainingData(String startTime, String endTime, String description, DataFormat dataFormat, Boolean coalesce, DataSource dataSource, Long seed, StatisticsConfig statisticsConfig, Map<String,String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter)	Create the metadata for a training dataset and save the corresponding training data into `location`.
Integer	FeatureViewBase.createTrainTestSplit(Float testSize, String trainStart, String trainEnd, String testStart, String testEnd, String description, DataFormat dataFormat, Boolean coalesce, DataSource dataSource, Long seed, StatisticsConfig statisticsConfig, Map<String,String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter)	Create the metadata for a training dataset and save the corresponding training data into `location`.
Integer	FeatureViewBase.createTrainValidationTestSplit(Float validationSize, Float testSize, String trainStart, String trainEnd, String validationStart, String validationEnd, String testStart, String testEnd, String description, DataFormat dataFormat, Boolean coalesce, DataSource dataSource, Long seed, StatisticsConfig statisticsConfig, Map<String,String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter)	Create the metadata for a training dataset and save the corresponding training data into `location`.
StreamFeatureGroup	FeatureStore.getOrCreateStreamFeatureGroup(@NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, DataSource dataSource, OnlineConfig onlineConfig)
abstract FeatureGroupBase	FeatureStoreBase.getOrCreateStreamFeatureGroup(@NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, DataSource dataSource, OnlineConfig onlineConfig)	Get a stream feature group metadata object or create a new one if it doesn't exist.
TrainingDatasetType	TrainingDatasetBase.getTrainingDatasetType(DataSource dataSource)
void	FeatureGroupBase.setDataSource(DataSource dataSource)
Map<String,String>	StorageConnector.AdlsConnector.sparkOptions(DataSource dataSource)
Map<String,String>	StorageConnector.BigqueryConnector.sparkOptions(DataSource dataSource)	Set spark options specific to BigQuery.
Map<String,String>	StorageConnector.GcsConnector.sparkOptions(DataSource dataSource)
Map<String,String>	StorageConnector.GlueConnector.sparkOptions(DataSource dataSource)
Map<String,String>	StorageConnector.HopsFsConnector.sparkOptions(DataSource dataSource)
Map<String,String>	StorageConnector.JdbcConnector.sparkOptions(DataSource dataSource)
Map<String,String>	StorageConnector.KafkaConnector.sparkOptions(DataSource dataSource)
Map<String,String>	StorageConnector.MongoDbConnector.sparkOptions(DataSource dataSource)
Map<String,String>	StorageConnector.RedshiftConnector.sparkOptions(DataSource dataSource)
Map<String,String>	StorageConnector.S3Connector.sparkOptions(DataSource dataSource)
Map<String,String>	StorageConnector.SapHanaConnector.sparkOptions(DataSource dataSource)
Map<String,String>	StorageConnector.SnowflakeConnector.sparkOptions(DataSource dataSource)
abstract Map<String,String>	StorageConnector.sparkOptions(DataSource dataSource)
Map<String,String>	StorageConnector.SqlConnector.sparkOptions(DataSource dataSource)

Constructors in com.logicalclocks.hsfs with parameters of type DataSource

Modifier	Constructor	Description
	TrainingDatasetBase(Integer version, String description, DataFormat dataFormat, Boolean coalesce, StorageConnector storageConnector, String location, List<Split> splits, String trainSplit, Long seed, FeatureStoreBase featureStore, StatisticsConfig statisticsConfig, List<String> label, String eventStartTime, String eventEndTime, TrainingDatasetType trainingDatasetType, Float validationSize, Float testSize, String trainStart, String trainEnd, String validationStart, String validationEnd, String testStart, String testEnd, Integer timeSplitSize, FilterLogic extraFilterLogic, Filter extraFilter, DataSource dataSource)

Uses of DataSource in com.logicalclocks.hsfs.beam

Methods in com.logicalclocks.hsfs.beam with parameters of type DataSource

Modifier and Type	Method	Description
StreamFeatureGroup	FeatureStore.createStreamFeatureGroup(@NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, DataSource dataSource)
StreamFeatureGroup	FeatureStore.getOrCreateStreamFeatureGroup(@NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, DataSource dataSource, OnlineConfig onlineConfig)

Uses of DataSource in com.logicalclocks.hsfs.beam.engine

Methods in com.logicalclocks.hsfs.beam.engine with parameters of type DataSource

Modifier and Type	Method	Description
StreamFeatureGroup	FeatureGroupEngine.getOrCreateFeatureGroup(FeatureStore featureStore, @NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, DataSource dataSource, OnlineConfig onlineConfig)

Uses of DataSource in com.logicalclocks.hsfs.engine

Methods in com.logicalclocks.hsfs.engine with parameters of type DataSource

Modifier and Type	Method	Description
StreamFeatureGroup	FeatureGroupEngine.getOrCreateFeatureGroup(FeatureStore featureStore, @NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, DataSource dataSource, OnlineConfig onlineConfig)

Uses of DataSource in com.logicalclocks.hsfs.flink

Methods in com.logicalclocks.hsfs.flink with parameters of type DataSource

Modifier and Type	Method	Description
StreamFeatureGroup	FeatureStore.createStreamFeatureGroup(@NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, DataSource dataSource)
StreamFeatureGroup	FeatureStore.getOrCreateStreamFeatureGroup(@NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, DataSource dataSource, OnlineConfig onlineConfig)

Uses of DataSource in com.logicalclocks.hsfs.flink.engine

Methods in com.logicalclocks.hsfs.flink.engine with parameters of type DataSource

Modifier and Type	Method	Description
StreamFeatureGroup	FeatureGroupEngine.getOrCreateFeatureGroup(FeatureStore featureStore, @NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, DataSource dataSource, OnlineConfig onlineConfig)

Uses of DataSource in com.logicalclocks.hsfs.spark

Methods in com.logicalclocks.hsfs.spark with parameters of type DataSource

Modifier and Type	Method	Description
StreamFeatureGroup	FeatureStore.createStreamFeatureGroup(@NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKey, List<String> partitionKey, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, DataSource dataSource)
Integer	FeatureView.createTrainingData(String startTime, String endTime, String description, DataFormat dataFormat, Boolean coalesce, DataSource dataSource, Long seed, StatisticsConfig statisticsConfig, Map<String,String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter)	Create the metadata for a training dataset and save the corresponding training data into `location`.
Integer	FeatureView.createTrainTestSplit(Float testSize, String trainStart, String trainEnd, String testStart, String testEnd, String description, DataFormat dataFormat, Boolean coalesce, DataSource dataSource, Long seed, StatisticsConfig statisticsConfig, Map<String,String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter)	Create the metadata for a training dataset and save the corresponding training data into `location`.
Integer	FeatureView.createTrainValidationTestSplit(Float validationSize, Float testSize, String trainStart, String trainEnd, String validationStart, String validationEnd, String testStart, String testEnd, String description, DataFormat dataFormat, Boolean coalesce, DataSource dataSource, Long seed, StatisticsConfig statisticsConfig, Map<String,String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter)	Create the metadata for a training dataset and save the corresponding training data into `location`.
StreamFeatureGroup	FeatureStore.getOrCreateStreamFeatureGroup(@NonNull String name, Integer version, String description, Boolean onlineEnabled, TimeTravelFormat timeTravelFormat, List<String> primaryKeys, List<String> partitionKeys, String eventTime, String hudiPrecombineKey, List<Feature> features, StatisticsConfig statisticsConfig, DataSource dataSource, OnlineConfig onlineConfig)

Constructors in com.logicalclocks.hsfs.spark with parameters of type DataSource

Modifier	Constructor	Description
	TrainingDataset(Integer version, String description, DataFormat dataFormat, Boolean coalesce, StorageConnector storageConnector, String location, List<Split> splits, String trainSplit, Long seed, FeatureStoreBase featureStore, StatisticsConfig statisticsConfig, List<String> label, String eventStartTime, String eventEndTime, TrainingDatasetType trainingDatasetType, Float validationSize, Float testSize, String trainStart, String trainEnd, String validationStart, String validationEnd, String testStart, String testEnd, Integer timeSplitSize, FilterLogic extraFilterLogic, Filter extraFilter, DataSource dataSource)

Uses of DataSource in com.logicalclocks.hsfs.spark.engine

Methods in com.logicalclocks.hsfs.spark.engine with parameters of type DataSource

Modifier and Type	Method	Description
StreamFeatureGroup	FeatureGroupEngine.getOrCreateStreamFeatureGroup(FeatureStore featureStore, String name, Integer version, String description, List<String> primaryKeys, List<String> partitionKeys, String hudiPrecombineKey, boolean onlineEnabled, StatisticsConfig statisticsConfig, String eventTime, TimeTravelFormat timeTravelFormat, List<Feature> features, DataSource dataSource, OnlineConfig onlineConfig)

Uses of DataSource in com.logicalclocks.hsfs.spark.util

Methods in com.logicalclocks.hsfs.spark.util with parameters of type DataSource

Modifier and Type	Method	Description
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	StorageConnectorUtils.read(StorageConnector.AdlsConnector connector, DataSource dataSource, String dataFormat, Map<String,String> options)	Reads path into a spark dataframe using the AdlsConnector.
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	StorageConnectorUtils.read(StorageConnector.BigqueryConnector connector, DataSource dataSource, Map<String,String> options)	Reads a query or a path into a spark dataframe using the sBigqueryConnector.
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	StorageConnectorUtils.read(StorageConnector.GcsConnector connector, DataSource dataSource, String dataFormat, Map<String,String> options)	Reads a path into a spark dataframe using the GcsConnector.
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	StorageConnectorUtils.read(StorageConnector.GlueConnector connector, DataSource dataSource, String dataFormat, Map<String,String> options)	Reads a Glue catalog table into a spark dataframe using the GlueConnector.
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	StorageConnectorUtils.read(StorageConnector.HopsFsConnector connector, DataSource dataSource, String dataFormat, Map<String,String> options)	Reads path into a spark dataframe using the HopsFsConnector.
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	StorageConnectorUtils.read(StorageConnector.JdbcConnector connector, DataSource dataSource, Map<String,String> options)	Reads query into a spark dataframe using the JdbcConnector.
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	StorageConnectorUtils.read(StorageConnector.RedshiftConnector connector, DataSource dataSource, Map<String,String> options)	Reads query into a spark dataframe using the RedshiftConnector.
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	StorageConnectorUtils.read(StorageConnector.S3Connector connector, DataSource dataSource, String dataFormat, Map<String,String> options)	Reads path into a spark dataframe using the S3Connector.
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	StorageConnectorUtils.read(StorageConnector.SnowflakeConnector connector, DataSource dataSource, Map<String,String> options)	Reads query into a spark dataframe using the SnowflakeConnector.
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	StorageConnectorUtils.read(StorageConnector.SqlConnector connector, DataSource dataSource, Map<String,String> options)	Reads a query into a spark dataframe using the SqlConnector.
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	StorageConnectorUtils.read(StorageConnector connector, DataSource dataSource, String dataFormat, Map<String,String> options)	Reads a query or a path into a spark dataframe using the storage connector.