Skip to content

Storage Connector#

Storage connectors encapsulate all information needed for the execution engine to read and write to specific storage. This storage can be S3, a JDBC compliant database or the distributed filesystem HOPSFS.

Retrieval#

[source]

get_storage_connector#

FeatureStore.get_storage_connector(name, connector_type)

Get a previously created storage connector from the feature store.

Storage connectors encapsulate all information needed for the execution engine to read and write to specific storage. This storage can be S3, a JDBC compliant database or the distributed filesystem HOPSFS.

If you want to connect to the online feature store, see the get_online_storage_connector method to get the JDBC connector for the Online Feature Store.

Getting a Storage Connector

sc = fs.get_storage_connector("demo_fs_meb10000_Training_Datasets", "HOPSFS")

td = fs.create_training_dataset(..., storage_connector=sc, ...)

Arguments

  • name str: Name of the storage connector to retrieve.
  • connector_type str: Type of the storage connector, e.g. "JDBC", "HOPSFS" or "S3".

Returns

StorageConnector. Storage connector object.


[source]

get_online_storage_connector#

FeatureStore.get_online_storage_connector()

Get the storage connector for the Online Feature Store of the respective project's feature store.

The returned storage connector depends on the project that you are connected to.

Returns

StorageConnector. JDBC storage connector to the Online Feature Store.


Properties#

[source]

access_key#

Access key for S3 buckets.


[source]

arguments#

Additional JDBC arguments.


[source]

connection_string#

JDBC connection string.


[source]

connector_type#

Type of the connector. S3, JDBC or HOPSFS.


[source]

id#

Id of the storage connector uniquely identifying it in the Feature store.


[source]

secret_key#

Secret key for S3 buckets.


[source]

server_encryption_algorithm#

Encryption algorithm if server-side S3 bucket encryption is enabled.


[source]

server_encryption_key#

Encryption key if server-side S3 bucket encryption is enabled.


Methods#

[source]

spark_options#

StorageConnector.spark_options()

Return prepared options to be passed to Spark, based on the additional arguments.


[source]

to_dict#

StorageConnector.to_dict()