Job#

hsfs.core.job.Job(
    id,
    name,
    creation_time,
    config,
    job_type,
    creator,
    executions=None,
    type=None,
    href=None,
    expand=None,
    items=None,
    count=None,
    **kwargs
)

Methods#

[source]

get_state#

Job.get_state()

Get the state of the job.

Returns

state. Current state of the job, which can be one of the following: INITIALIZING, INITIALIZATION_FAILED, FINISHED, RUNNING, ACCEPTED, FAILED, KILLED, NEW, NEW_SAVING, SUBMITTED, AGGREGATING_LOGS, FRAMEWORK_FAILURE, STARTING_APP_MASTER, APP_MASTER_START_FAILED, GENERATING_SECURITY_MATERIAL, CONVERTING_NOTEBOOK

[source]

get_final_state#

Job.get_final_state()

Get the final state of the job.

Returns

final_state. Final state of the job, which can be one of the following: UNDEFINED, FINISHED, FAILED, KILLED, FRAMEWORK_FAILURE, APP_MASTER_START_FAILED, INITIALIZATION_FAILED. UNDEFINED indicates that the job is still running.

Job Configuration#

[source]

JobConfiguration#

hsfs.core.job_configuration.JobConfiguration(
    am_memory=2048,
    am_cores=1,
    executor_memory=4096,
    executor_cores=1,
    executor_instances=1,
    dynamic_allocation=True,
    dynamic_min_executors=1,
    dynamic_max_executors=2,
    **kwargs
)