hopsworks.core.trino_catalog_api #

TrinoCatalogApi #

Trino catalogs of the current project, and the query engine restart that loads them.

A catalog makes an external source queryable from the query engine. Creating one records the change; it becomes queryable once the query engine restarts, which happens on the cluster's schedule or immediately when a cluster administrator asks for it.

Returned by

Project.get_trino_catalog_api

create_catalog #

create_catalog(
    name: str,
    connector_type: str,
    properties: dict[str, str],
    data_source_name: str | None = None,
    featurestore_id: int | None = None,
) -> dict[str, Any]

Create a Trino catalog.

The catalog is recorded now and becomes queryable when the query engine restarts. Check get_capabilities()["nextScheduledRestart"] for when that is, or call restart() as a cluster administrator to do it immediately.

PARAMETER	DESCRIPTION
`name`	Full catalog name. It must start with `<project>__` in lowercase. TYPE: `str`
`connector_type`	A Trino connector from `get_capabilities()["connectors"]`. TYPE: `str`
`properties`	Connector properties, without `connector.name`. A value may reference a Hopsworks secret as `${HOPSWORKS_SECRET:<name>}`, or a credential-file bundle as `${HOPSWORKS_MOUNT:<bundle>}`, instead of holding the value itself. TYPE: `dict[str, str]`
`data_source_name`	Derive credential properties from this data source, for a catalog built from `get_catalog_template`. Leave unset for a hand-written catalog. TYPE: `str \| None` DEFAULT: `None`
`featurestore_id`	The data source's feature store. Required with `data_source_name`. TYPE: `int \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`dict[str, Any]`	The created catalog, including the `status` it is waiting in.

RAISES	DESCRIPTION
`ValueError`	If only one of `data_source_name` and `featurestore_id` is passed.
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.

delete_catalog #

delete_catalog(name: str) -> None

Delete a catalog.

The catalog stays queryable until the query engine restarts and unloads it.

PARAMETER	DESCRIPTION
`name`	The catalog's full name, including the `<project>__` prefix. TYPE: `str`

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.

get_capabilities #

get_capabilities() -> dict[str, Any]

What this cluster's query engine supports, and when it next restarts.

RETURNS	DESCRIPTION
`dict[str, Any]`	`connectors` a catalog may use, `testConnectionAvailable` for whether a live connection
`dict[str, Any]`	test is possible, `restarting` while the engine is mid-restart, `nextScheduledRestart`
`dict[str, Any]`	as the instant a pending catalog next goes live (null when no restart is scheduled), and
`dict[str, Any]`	`mountableSecretsAvailable` for whether credential files can be delivered.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.

get_catalog #

get_catalog(name: str) -> dict[str, Any]

One catalog with its properties, for inspection or editing.

Secret-bearing property values come back masked (********). Submitting a masked value unchanged through update_catalog keeps the stored secret, so a masked property does not have to be retyped to change a different one.

PARAMETER	DESCRIPTION
`name`	The catalog's full name, including the `<project>__` prefix. TYPE: `str`

RETURNS	DESCRIPTION
`dict[str, Any]`	The catalog: its `name`, `connectorType`, `status`, pending `operation`, and
`dict[str, Any]`	`properties` with secret-bearing values masked.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.

get_catalog_template #

get_catalog_template(
    data_source_name: str, featurestore_id: int
) -> dict[str, Any]

A proposed catalog derived from a data source, to be reviewed and then created.

Nothing is created or written by asking for this. Credential properties come back as a reference to a Hopsworks secret or credential-file bundle rather than a value: the value is read from the data source when the catalog is created, so no credential is sent here.

PARAMETER	DESCRIPTION
`data_source_name`	Name of the data source (storage connector) to derive from. TYPE: `str`
`featurestore_id`	The feature store the data source belongs to. TYPE: `int`

RETURNS	DESCRIPTION
`dict[str, Any]`	`supported` and, when false, `reason` saying why the source cannot be mapped. When true:
`dict[str, Any]`	`suggestedName`, `connectorType`, and `properties`.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.

get_catalogs #

get_catalogs() -> list[dict[str, Any]]

The project's Trino catalogs, plus the cluster's shared default catalogs.

RETURNS	DESCRIPTION
`list[dict[str, Any]]`	One entry per catalog: its `name`, `connectorType`, `status`, pending `operation`,
`list[dict[str, Any]]`	`creator`, and `defaultCatalog` for a cluster-wide one. Properties are not included;
`list[dict[str, Any]]`	use `get_catalog` for those.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.

restart #

restart() -> dict[str, Any]

Apply every pending catalog change now, instead of waiting for the schedule.

Requires cluster administrator (HOPS_ADMIN) privileges; the backend refuses otherwise. This interrupts queries running anywhere on the cluster, so prefer waiting for get_capabilities()["nextScheduledRestart"] unless the change is needed sooner.

Pending changes are written and the query engine is then restarted, in that order: a restart on its own would load nothing, because a newly created catalog is only a record until it is written.

RETURNS	DESCRIPTION
`dict[str, Any]`	`restarted` false when the restart was skipped as unnecessary, and `quarantined` naming
`dict[str, Any]`	any catalog removed because the engine could not load it. A quarantined catalog is not
`dict[str, Any]`	applied; its own page carries the load error.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the user is not a cluster administrator, or the backend encounters an error when handling the request.

test_connection #

test_connection(
    name: str,
    connector_type: str,
    properties: dict[str, str],
    data_source_name: str | None = None,
    featurestore_id: int | None = None,
) -> None

Check that a catalog definition can actually reach its source, without creating it.

Available only where get_capabilities()["testConnectionAvailable"] is true, which needs the cluster's optional test coordinator. Returns nothing on success and raises with the engine's own message on failure, so a bad host or credential is reported before the catalog exists.

PARAMETER	DESCRIPTION
`name`	Full catalog name, as it would be created. TYPE: `str`
`connector_type`	A Trino connector from `get_capabilities()["connectors"]`. TYPE: `str`
`properties`	The connector properties to test, as they would be stored. TYPE: `dict[str, str]`
`data_source_name`	Resolve credential properties from this data source, exactly as `create_catalog` would. Leave unset for a hand-written catalog. TYPE: `str \| None` DEFAULT: `None`
`featurestore_id`	The data source's feature store. Required with `data_source_name`. TYPE: `int \| None` DEFAULT: `None`

RAISES	DESCRIPTION
`ValueError`	If only one of `data_source_name` and `featurestore_id` is passed.
`hopsworks.client.exceptions.RestAPIError`	If the connection fails, or the backend encounters an error when handling the request.

update_catalog #

update_catalog(
    name: str,
    connector_type: str,
    properties: dict[str, str],
) -> dict[str, Any]

Change an existing catalog's connector or properties.

Like a create, the change takes effect at the next query engine restart. A property left at its masked value keeps the secret already stored for it, which is how an update changes one property without retyping the others' credentials.

PARAMETER	DESCRIPTION
`name`	The catalog's full name, including the `<project>__` prefix. TYPE: `str`
`connector_type`	The Trino connector the catalog uses. TYPE: `str`
`properties`	The full set of connector properties to store, as from `get_catalog`. TYPE: `dict[str, str]`

RETURNS	DESCRIPTION
`dict[str, Any]`	The updated catalog, including the `status` it is waiting in.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.