Executions API#
Creation#
run#
Job.run(args=None, await_termination=False)
Run the job, with the option of passing runtime arguments.
Example of a blocking execution and downloading logs once execution is finished.
# Run the job
execution = job.run(await_termination=True)
# True if job executed successfully
print(execution.success)
# Download logs
out_log_path, err_log_path = execution.download_logs()
- args
Optional[str]: optional runtime arguments for the job - await_termination
bool: if True wait until termination is complete
Returns
Execution. The execution object for the submitted run.
Retrieval#
get_executions#
Job.get_executions()
Retrieves all executions for the job ordered by submission time.
Returns
List[Execution]
Raises
RestAPIError in case the backend fails to retrieve executions.
Properties#
app_id#
Application id for the execution
args#
Arguments set for the execution.
duration#
Duration in milliseconds the execution ran.
final_status#
Final status of the execution. Can be UNDEFINED, SUCCEEDED, FAILED or KILLED.
hdfs_user#
Filesystem user for the execution.
id#
Id of the execution
job_name#
Name of the job the execution belongs to
job_type#
Type of the job the execution belongs to
progress#
Progress of the execution.
state#
Current state of the execution.
stderr_path#
Path in Hopsworks Filesystem to stderr log file
stdout_path#
Path in Hopsworks Filesystem to stdout log file
submission_time#
Timestamp when the execution was submitted
success#
Boolean to indicate if execution ran successfully or failed
Returns
bool. True if execution ran successfully. False if execution failed or was killed.
user#
User that submitted the execution.
Methods#
await_termination#
Execution.await_termination()
Wait until execution reaches terminal state Raises
RestAPIError.
delete#
Execution.delete()
Delete the execution
Potentially dangerous operation
This operation deletes the execution.
Raises
RestAPIError.
download_logs#
Execution.download_logs(path=None)
Download stdout and stderr logs for the execution Example for downloading and printing the logs
# Download logs
out_log_path, err_log_path = execution.download_logs()
out_fd = open(out_log_path, "r")
print(out_fd.read())
err_fd = open(err_log_path, "r")
print(err_fd.read())
Arguments
- path: path to download the logs. must be
str
Returns
str. Path to downloaded log for stdout.
str. Path to downloaded log for stderr.
get_url#
Execution.get_url()