hopsworks.execution #

Execution #

For backwards compatibility hopsworks.execution.Execution is still available as hsfs.core.execution.Execution. The use of this alias is discouraged as it is to be deprecated.

Returned by

app_id `property` #

Application id for the execution.

app_url `property` #

app_url: str | None

URL to the Python App UI (Streamlit) if the execution is running.

Returns the full URL to access the Streamlit application through the Hopsworks proxy, or None if the execution is not running or the app URL is not available.

args `property` #

Arguments set for the execution.

duration `property` #

Duration in milliseconds the execution ran.

final_status `property` #

Final status of the execution. Can be UNDEFINED, SUCCEEDED, FAILED or KILLED.

hdfs_user `property` #

Filesystem user for the execution.

id `property` #

Id of the execution.

job_name `property` #

Name of the job the execution belongs to.

job_type `property` #

Type of the job the execution belongs to.

progress `property` #

Progress of the execution.

state `property` #

Current state of the execution.

Can be: INITIALIZING, INITIALIZATION_FAILED, FINISHED, RUNNING, ACCEPTED, FAILED, KILLED, NEW, NEW_SAVING, SUBMITTED, AGGREGATING_LOGS, FRAMEWORK_FAILURE, STARTING_APP_MASTER, APP_MASTER_START_FAILED, GENERATING_SECURITY_MATERIAL, or CONVERTING_NOTEBOOK.

stderr_path `property` #

Path in Hopsworks Filesystem to stderr log file.

stdout_path `property` #

Path in Hopsworks Filesystem to stdout log file.

submission_time `property` #

Timestamp when the execution was submitted.

success `property` #

success: bool | None

Boolean to indicate if execution ran successfully or failed.

user `property` #

User that submitted the execution.

await_termination #

await_termination(timeout: float | None = None)

Wait until execution terminates.

PARAMETER	DESCRIPTION
`timeout`	The maximum waiting time in seconds. If `None` the waiting time is unbounded. Note: the actual waiting time may be bigger by approximately 3 seconds. TYPE: `float \| None` DEFAULT: `None`

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.
`hopsworks.client.exceptions.JobExecutionException`	If the execution finished with a failure status.

delete #

delete()

Delete the execution.

Potentially dangerous operation

This operation deletes the execution.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.

download_logs #

download_logs(
    path: str | None = None,
) -> tuple[str | None, str | None]

Download stdout and stderr logs for the execution.

Downloading and printing the logs

# Download logs
out_log_path, err_log_path = execution.download_logs()

out_fd = open(out_log_path, "r")
print(out_fd.read())

err_fd = open(err_log_path, "r")
print(err_fd.read())

PARAMETER	DESCRIPTION
`path`	Path to download the logs. TYPE: `str \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`stdout`	Path to downloaded log for stdout. TYPE: `str \| None`
`stderr`	Path to downloaded log for stderr. TYPE: `str \| None`

get_pod_logs #

get_pod_logs(
    table_index: int | None = None,
    lines: int | None = None,
    limit_bytes: int | None = None,
) -> execution_pod_log.ExecutionPodLog

Read the live log of the execution's pod, or of one table's pod.

This streams the running pod, so it shows progress while the execution is still going, unlike Execution.download_logs which fetches the archived files. The returned ExecutionPodLog.status tells you whether the log is available yet.

PARAMETER	DESCRIPTION
`table_index`	For a multi-table ingestion, the index of the table whose pod to read; `None` reads the execution's own pod. TYPE: `int \| None` DEFAULT: `None`
`lines`	Number of log lines to tail from the end. TYPE: `int \| None` DEFAULT: `None`
`limit_bytes`	Maximum number of bytes to read from the pod log stream. TYPE: `int \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`execution_pod_log.ExecutionPodLog`	The pod log, including its availability status and content.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.

get_url #

get_url()

Get url to view execution details in Hopsworks UI.

stop #

stop()

Stop the execution.

Potentially dangerous operation

This operation stops the execution.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.

stop_table #

stop_table(table_index: int) -> None

Stop the ingestion of a single table of a multi-table ingestion execution.

The other tables keep running. Use Execution.stop to stop the whole execution instead. To exclude a table from future runs rather than stopping the current one, disable it on the job with MultiTableIngestionJob.set_table_enabled.

PARAMETER	DESCRIPTION
`table_index`	Index of the table to stop, matching its position in the job's target list. TYPE: `int`

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.

hopsworks.execution #

Execution #

app_id property #

app_url property #

args property #

duration property #

final_status property #

hdfs_user property #

id property #

job_name property #

job_type property #

progress property #

state property #

stderr_path property #

stdout_path property #

submission_time property #

success property #

user property #