Uses of Class
com.logicalclocks.hsfs.StorageConnector

Packages that use StorageConnector

Package	Description
com.logicalclocks.hsfs
com.logicalclocks.hsfs.flink
com.logicalclocks.hsfs.spark
com.logicalclocks.hsfs.spark.engine
com.logicalclocks.hsfs.spark.util

Uses of StorageConnector in com.logicalclocks.hsfs

Subclasses of StorageConnector in com.logicalclocks.hsfs

Modifier and Type	Class and Description
static class	StorageConnector.AdlsConnector
static class	StorageConnector.BigqueryConnector
static class	StorageConnector.GcsConnector
static class	StorageConnector.HopsFsConnector
static class	StorageConnector.JdbcConnector
static class	StorageConnector.KafkaConnector
static class	StorageConnector.RedshiftConnector
static class	StorageConnector.S3Connector
static class	StorageConnector.SnowflakeConnector

Fields in com.logicalclocks.hsfs declared as StorageConnector

Modifier and Type	Field and Description
protected StorageConnector	TrainingDatasetBase.storageConnector

Methods in com.logicalclocks.hsfs that return StorageConnector

Modifier and Type	Method and Description
abstract StorageConnector	FeatureStoreBase.getStorageConnector(String name) Get a list of all versions of an external feature group from the feature store.
StorageConnector	StorageConnector.refetch()

Methods in com.logicalclocks.hsfs with parameters of type StorageConnector

Modifier and Type	Method and Description
TrainingDatasetType	TrainingDatasetBase.getTrainingDatasetType(StorageConnector storageConnector)

Uses of StorageConnector in com.logicalclocks.hsfs.flink

Methods in com.logicalclocks.hsfs.flink that return StorageConnector

Modifier and Type	Method and Description
StorageConnector	FeatureStore.getStorageConnector(String name)

Uses of StorageConnector in com.logicalclocks.hsfs.spark

Methods in com.logicalclocks.hsfs.spark that return StorageConnector

Modifier and Type	Method and Description
StorageConnector	FeatureStore.getStorageConnector(String name) Get a previously created storage connector from the feature store.

Methods in com.logicalclocks.hsfs.spark with parameters of type StorageConnector

Modifier and Type	Method and Description
Integer	FeatureView.createTrainingData(String startTime, String endTime, String description, DataFormat dataFormat, Boolean coalesce, StorageConnector storageConnector, String location, Long seed, StatisticsConfig statisticsConfig, Map<String,String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter) Create the metadata for a training dataset and save the corresponding training data into `location`.
Integer	FeatureView.createTrainTestSplit(Float testSize, String trainStart, String trainEnd, String testStart, String testEnd, String description, DataFormat dataFormat, Boolean coalesce, StorageConnector storageConnector, String location, Long seed, StatisticsConfig statisticsConfig, Map<String,String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter) Create the metadata for a training dataset and save the corresponding training data into `location`.
Integer	FeatureView.createTrainValidationTestSplit(Float validationSize, Float testSize, String trainStart, String trainEnd, String validationStart, String validationEnd, String testStart, String testEnd, String description, DataFormat dataFormat, Boolean coalesce, StorageConnector storageConnector, String location, Long seed, StatisticsConfig statisticsConfig, Map<String,String> writeOptions, FilterLogic extraFilterLogic, Filter extraFilter) Create the metadata for a training dataset and save the corresponding training data into `location`.

Constructors in com.logicalclocks.hsfs.spark with parameters of type StorageConnector

Constructor and Description
ExternalFeatureGroup(FeatureStore featureStore, @NonNull String name, Integer version, String query, ExternalDataFormat dataFormat, String path, Map<String,String> options, @NonNull StorageConnector storageConnector, String description, List<String> primaryKeys, List<Feature> features, StatisticsConfig statisticsConfig, String eventTime, boolean onlineEnabled, String onlineTopicName)
TrainingDataset(@NonNull String name, Integer version, String description, DataFormat dataFormat, Boolean coalesce, StorageConnector storageConnector, String location, List<Split> splits, String trainSplit, Long seed, FeatureStore featureStore, StatisticsConfig statisticsConfig, List<String> label, String eventStartTime, String eventEndTime, TrainingDatasetType trainingDatasetType, Float validationSize, Float testSize, String trainStart, String trainEnd, String validationStart, String validationEnd, String testStart, String testEnd, Integer timeSplitSize, FilterLogic extraFilterLogic, Filter extraFilter)

Uses of StorageConnector in com.logicalclocks.hsfs.spark.engine

Methods in com.logicalclocks.hsfs.spark.engine with parameters of type StorageConnector

Modifier and Type	Method and Description
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	SparkEngine.read(StorageConnector storageConnector, String dataFormat, Map<String,String> readOptions, String location)
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	SparkEngine.readStream(StorageConnector storageConnector, String dataFormat, String messageFormat, String schema, Map<String,String> options, boolean includeMetadata)
void	SparkEngine.setupConnectorHadoopConf(StorageConnector storageConnector)

Uses of StorageConnector in com.logicalclocks.hsfs.spark.util

Methods in com.logicalclocks.hsfs.spark.util with parameters of type StorageConnector

Modifier and Type	Method and Description
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	StorageConnectorUtils.read(StorageConnector connector, String query, String dataFormat, Map<String,String> options, String path) Reads a query or a path into a spark dataframe using the storage connector.