hopsworks.core.semantic_layer_api #
SemanticLayerApi #
The semantic projects of a Hopsworks project: metric definitions compiled from YAML and queried as metrics.
A semantic project reads dbt or MetricFlow semantic YAML from a HopsFS directory, usually a dbt project in a Hopsworks git repository. Its semantic models are feature groups, and its queries run on Trino as the caller, so members see only the data they can already read. Each SemanticProject compiles, lists its metrics and dimensions, and turns metric queries into SQL or rows.
Register and query a dbt project
Returned by
create_project #
create_project(
name: str,
root_path: str,
git_repository_id: int | None = None,
format: Literal[
"AUTO", "DBT", "NATIVE", "MANIFEST_JSON"
] = "AUTO",
deployment_mode: Literal["FOLLOW", "PINNED"] = "FOLLOW",
tags: dict[str, Any] | None = None,
) -> semantic_layer.SemanticProject
Register a directory of semantic definitions, which starts its first compile in the background.
| PARAMETER | DESCRIPTION |
|---|---|
name | Name of the semantic project, unique in the Hopsworks project, lowercase letters, digits and underscores starting with a letter. TYPE: |
root_path | Directory of the definitions, relative to the project root, under TYPE: |
git_repository_id | Hopsworks git repository holding TYPE: |
format |
TYPE: |
deployment_mode |
TYPE: |
tags | Schematised tags to attach in the same request, tag schema name to value, such as |
| RETURNS | DESCRIPTION |
|---|---|
semantic_layer.SemanticProject | The semantic project. |
| RAISES | DESCRIPTION |
|---|---|
hopsworks.client.exceptions.RestAPIError | If the name is taken or invalid, the path is outside the project, a tag is unknown, invalid or a missing mandatory one, or the backend encounters another error. |
create_time_spine #
create_time_spine(
start: str | datetime.date,
end: str | datetime.date,
granularity: Literal[
"second",
"minute",
"hour",
"day",
"week",
"month",
"quarter",
"year",
] = "day",
custom_granularities: Mapping[
str, str | Callable[[Any], Any]
]
| None = None,
) -> FeatureGroup
Write the time spine that cumulative metrics, offsets, join_to_timespine and custom granularities join to.
The spine is the offline feature group time_spine_<granularity>, version 1, with one row per period from start through end. Its primary key date_<granularity> holds the start of each period: a date for a day or a coarser grain, a timestamp below a day. A semantic project's time spine model at that grain binds the feature group by this name, unless its config.meta.hopsworks.feature_group names another. When the feature group exists the rows are inserted into it, so a wider range extends the spine, as long as the custom granularities are the ones it already has. The spine holds one row per period, so a fine grain over a long range is large: ten years are 3,653 days but 315 million seconds.
A daily spine with a fiscal quarter and a fiscal year
| PARAMETER | DESCRIPTION |
|---|---|
start | First day or instant of the spine, as an ISO-8601 string, a date or a datetime; it is moved back to the start of its period. |
end | Last day or instant of the spine, included. |
granularity | Grain of the spine; weeks start on Monday. TYPE: |
custom_granularities | One column per custom granularity the time spine model declares, column name to definition. A string is a pandas period frequency, such as TYPE: |
| RETURNS | DESCRIPTION |
|---|---|
FeatureGroup | The feature group, with the rows written. |
| RAISES | DESCRIPTION |
|---|---|
ValueError | For a granularity not listed, a |
hopsworks.client.exceptions.RestAPIError | If the backend fails to create or write the feature group, for example when an existing spine has other columns. |
delete_project #
delete_project(name: str) -> None
Delete a semantic project and its builds; the definitions in HopsFS are left in place.
| PARAMETER | DESCRIPTION |
|---|---|
name | Name of the semantic project. TYPE: |
| RAISES | DESCRIPTION |
|---|---|
hopsworks.client.exceptions.SemanticLayerException | If the Hopsworks project has no semantic project with that name. |
hopsworks.client.exceptions.RestAPIError | If the backend encounters an error when handling the request. |
get_project #
get_project(
name: str,
) -> semantic_layer.SemanticProject | None
A semantic project by name.
| PARAMETER | DESCRIPTION |
|---|---|
name | Name of the semantic project. TYPE: |
| RETURNS | DESCRIPTION |
|---|---|
semantic_layer.SemanticProject | None | The semantic project, or |
| RAISES | DESCRIPTION |
|---|---|
hopsworks.client.exceptions.RestAPIError | If the backend encounters an error when handling the request. |
get_projects #
get_projects() -> list[semantic_layer.SemanticProject]
Every semantic project of the Hopsworks project.
| RETURNS | DESCRIPTION |
|---|---|
list[semantic_layer.SemanticProject] | The semantic projects, each with its newest build. |
| RAISES | DESCRIPTION |
|---|---|
hopsworks.client.exceptions.RestAPIError | If the backend encounters an error when handling the request. |