Skip to content

Compute Guides#

Compute is where code runs inside a project: notebooks, a terminal, jobs, schedules and served apps, all on the project's Python environments. The guides go from writing code interactively to running it on a schedule.

  • Start here

    Upload a script and run it as a job from any Python session. Inside the project, Jupyter and the terminal are already logged in.

    jobs_api = project.get_job_api()
config = jobs_api.get_configuration("PYTHON")
config["appPath"] = "Resources/script.py"
job = jobs_api.create_job("py_job", config)
execution = job.run(await_termination=True)

    Run a Python job · Jupyter · Python environments

Write code

  • Jupyter Python, PySpark and Ray notebooks running against the project.
  • Terminal A shell in the project with the CLI, Git and coding agents preinstalled.
  • Python environments Install libraries, clone and export environments, run custom build commands.
  • Apps Serve Streamlit, Gradio or any web app from the project.

Run and schedule

  • Jobs Run Python, notebook, PySpark, Spark or Ray code as a job, on demand or on a schedule.
  • Airflow Orchestrate jobs as DAGs with the Hopsworks operators.
  • Kubernetes scheduling Node selectors, tolerations and Kueue queues for where work lands.
  • Python deployments Expose a Python function behind a REST endpoint.