Executions API#

Creation#

[source]

run#

Job.run(args=None, await_termination=True)

Run the job.

Run the job, by default awaiting its completion, with the option of passing runtime arguments.

# connect to the Feature Store
fs = ...

# get the Feature Group instances
fg = fs.get_or_create_feature_group(...)

# insert in to feature group
job, _ = fg.insert(df, write_options={"start_offline_materialization": False})

# run job
execution = job.run()

# True if job executed successfully
print(execution.success)

# Download logs
out_log_path, err_log_path = execution.download_logs()

Arguments

• args str: Optional runtime arguments for the job.
• await_termination bool: Identifies if the client should wait for the job to complete, defaults to True.

Returns

Execution. The execution object for the submitted run.

Retrieval#

[source]

get_executions#

Job.get_executions()

Retrieves all executions for the job ordered by submission time.

Returns

List[Execution]

Raises

RestAPIError in case the backend fails to retrieve executions.

Properties#

[source]

app_id#

Application id for the execution

[source]

args#

Arguments set for the execution.

[source]

duration#

Duration in milliseconds the execution ran.

[source]

final_status#

Final status of the execution. Can be UNDEFINED, SUCCEEDED, FAILED or KILLED.

[source]

hdfs_user#

Filesystem user for the execution.

[source]

id#

Id of the execution

[source]

job_name#

Name of the job the execution belongs to

[source]

job_type#

Type of the job the execution belongs to

[source]

progress#

Progress of the execution.

[source]

state#

Current state of the execution.

Can be: INITIALIZING, INITIALIZATION_FAILED, FINISHED, RUNNING, ACCEPTED, FAILED, KILLED, NEW, NEW_SAVING, SUBMITTED, AGGREGATING_LOGS, FRAMEWORK_FAILURE, STARTING_APP_MASTER, APP_MASTER_START_FAILED, GENERATING_SECURITY_MATERIAL, or CONVERTING_NOTEBOOK.

[source]

stderr_path#

Path in Hopsworks Filesystem to stderr log file

[source]

stdout_path#

Path in Hopsworks Filesystem to stdout log file

[source]

submission_time#

Timestamp when the execution was submitted

[source]

success#

Boolean to indicate if execution ran successfully or failed

Returns

bool. True if execution ran successfully. False if execution failed or was killed.

[source]

user#

User that submitted the execution.

Methods#

[source]

await_termination#

Execution.await_termination(timeout=None)

Wait until execution terminates.

Arguments

• timeout float | None: the maximum waiting time in seconds, if None the waiting time is unbounded; defaults to None. Note: the actual waiting time may be bigger by approximately 3 seconds.

Raises

RestAPIError.

[source]

delete#

Execution.delete()

Delete the execution

Raises

RestAPIError.

[source]

download_logs#

Execution.download_logs(path=None)

Download stdout and stderr logs for the execution Example for downloading and printing the logs

# Download logs
out_log_path, err_log_path = execution.download_logs()

out_fd = open(out_log_path, "r")
print(out_fd.read())

err_fd = open(err_log_path, "r")
print(err_fd.read())

Arguments

• path: path to download the logs. must be str

Returns

str. Path to downloaded log for stdout. str. Path to downloaded log for stderr.

[source]

get_url#

Execution.get_url()

[source]

stop#

Execution.stop()

Stop the execution

Raises

RestAPIError.