hsfs.core.online_ingestion #

[source] OnlineIngestion #

Metadata object used to provide Online Ingestion information for a feature group.

This class encapsulates the state and results of an online ingestion operation, including progress tracking and log retrieval.

id: int | None

Get the unique identifier for the ingestion operation.

RETURNS	DESCRIPTION
`int \| None`	Optional[int]: The ingestion ID.

num_entries: int | None

Get the total number of entries to ingest.

RETURNS	DESCRIPTION
`int \| None`	Optional[int]: The number of entries.

results: list[online_ingestion_result.OnlineIngestionResult]

Get the list of ingestion results.

RETURNS	DESCRIPTION
`list[online_ingestion_result.OnlineIngestionResult]`	List[OnlineIngestionResult]: List of ingestion result objects.

feature_group: fg_mod.FeatureGroup

Get the feature group associated with this ingestion.

RETURNS	DESCRIPTION
`FeatureGroup`	The associated feature group. TYPE: `fg_mod.FeatureGroup`

refresh()

Refresh the state of this OnlineIngestion object from the backend.

wait_for_completion(options: dict[str, Any] = None)

Wait for the online ingestion operation to complete, displaying a progress bar.

PARAMETER	DESCRIPTION
`options`	Options for waiting. - "timeout" (int): Maximum time to wait in seconds (default: 60). - "period" (int): Polling period in seconds (default: 1). TYPE: `Dict[str, Any]` DEFAULT: `None`

RAISES	DESCRIPTION
`Warning`	If the timeout is exceeded before completion.

print_logs(priority: str = 'error', size: int = 20)

Print logs related to the online ingestion operation from OpenSearch.

PARAMETER	DESCRIPTION
`priority`	Log priority to filter by (default: "error"). TYPE: `str` DEFAULT: `'error'`
`size`	Number of log entries to retrieve (default: 20). TYPE: `int` DEFAULT: `20`