Skip to content

Online Ingestion#

[source]

OnlineIngestion#

hsfs.core.online_ingestion.OnlineIngestion(
    id=None, num_entries=None, results=None, feature_group=None, **kwargs
)

Metadata object used to provide Online Ingestion information for a feature group.

This class encapsulates the state and results of an online ingestion operation, including progress tracking and log retrieval.

Properties#

[source]

feature_group#

Get the feature group associated with this ingestion.

Returns

FeatureGroup: The associated feature group.

[source]

id#

Get the unique identifier for the ingestion operation.

Returns

Optional[int]: The ingestion ID.

[source]

num_entries#

Get the total number of entries to ingest.

Returns

Optional[int]: The number of entries.

[source]

results#

Get the list of ingestion results.

Returns

List[OnlineIngestionResult]: List of ingestion result objects.

Methods#

[source]

OnlineIngestion.print_logs(priority="error", size=20)

Print logs related to the online ingestion operation from OpenSearch.

Arguments

  • priority (str, optional): Log priority to filter by (default: "error").
  • size (int, optional): Number of log entries to retrieve (default: 20).

[source]

refresh#

OnlineIngestion.refresh()

Refresh the state of this OnlineIngestion object from the backend.

[source]

wait_for_completion#

OnlineIngestion.wait_for_completion(options=None)

Wait for the online ingestion operation to complete, displaying a progress bar.

Arguments

  • options (Dict[str, Any], optional): Options for waiting.
    • "timeout" (int): Maximum time to wait in seconds (default: 60).
    • "period" (int): Polling period in seconds (default: 1).

Raises

  • Warning: If the timeout is exceeded before completion.