How To Run A Python Notebook#

Introduction#

Jupyter is provided as a service in Hopsworks, providing the same user experience and features as if run on your laptop.

Supports JupyterLab and the classic Jupyter front-end
Configured with Python3, Spark, PySpark and SparkR kernels

Important

If Hopsworks is not configured to run Jupyter on Kubernetes then the Python kernel is disabled by default. In this case the Python kernel can be enabled by setting the configuration variable enable_jupyter_python_kernel_non_kubernetes to True. Follow this guide for instructions on how to set a configuration variable.

Step 1: Jupyter dashboard#

The image below shows the Jupyter service page in Hopsworks and is accessed by clicking Jupyter in the sidebar.

From this page, you can configure various options and settings to start Jupyter with as described in the sections below.

Step 2 (Optional): Configure resources#

Next step is to configure Jupyter, Click edit configuration to get to the configuration page and select Python.

Container cores: Number of cores to allocate for the Jupyter instance
Container memory: Number of MBs to allocate for the Jupyter instance

Configured resource pool is shared by all running kernels. If a kernel crashes while executing a cell, try increasing the Container memory.

Resource configuration for the Python kernel

Click Save to save the new configuration.

Step 3 (Optional): Configure max runtime and root path#

Before starting the server there are two additional configurations that can be set next to the Run Jupyter button.

The runtime of the Jupyter instance can be configured, this is useful to ensure that idle instances will not be hanging around and keep allocating resources. If a limited runtime is not desirable, this can be disabled by setting no limit.

The root path from which to start the Jupyter instance can be configured. By default it starts by setting the /Jupyter folder as the root.

Step 4: Start Jupyter#

Start the Jupyter instance by clicking the Run Jupyter button.

Starting Jupyter and running a Python notebook

Conclusion#

In this guide you learned how to configure and run a Python application in Jupyter. You can now follow this guide to install a library that can be used in a notebook.