Feature Monitoring#

Introduction#

Feature Monitoring complements the Hopsworks data validation capabilities by allowing you to monitor your data once they have been ingested into the Feature Store. Hopsworks feature monitoring user interface is centered around three functionalities:

• Scheduled Statistics: The user defines a detection window over its data for which Hopsworks will compute the statistics on a regular basis. The results are stored in Hopsworks and enable the user to visualise the temporal evolution of statistical metrics on its data. This can be enabled for a whole Feature Group or Feature View, or for a particular Feature. For more details, see the Scheduled statistics guide.

• Statistics Comparison: This variant allows the user to schedule the statistics computation on both a detection and a reference window, and compare them on a selected feature using a single scalar metric (e.g., the mean). By providing information about how to compare those statistics, you can setup alerts to quickly detect critical change in the data. For more details, see the Statistics comparison guide.

• Data Distribution Comparison: Instead of a single scalar metric, this variant compares the whole distribution of a feature between the detection and reference windows using distance metrics such as PSI or KL divergence. This helps detect changes in the shape of the data that a single metric might miss. For more details, see the Distribution comparison guide.

Define windows over feature data#

Windows define the boundaries of the feature data on which Hopsworks operates. Both statistics and data distributions are computed over the feature data delimited by a window. There are different types of windows depending on how they evolve over time. A window can have either a fixed length (e.g., static window) or variable length (e.g., expanding window). Moreover, windows can stick to a specific point in time (e.g., static window) or move over time (e.g., sliding or rolling window).

These types of windows apply to both detection and reference windows. Different types of windows allows for different use cases depending on whether you enable feature monitoring on your Feature Groups or Feature Views.

See more details about detection and reference windows in the Detection windows and Reference windows guides.

Visualize metrics on a time series#

Hopsworks provides an interactive graph to make the exploration of statistics and metrics (e.g., distribution-based distances) more efficient and help you find unexpected trends or anomalous values faster. See the Interactive graph guide for more information.

Alerting#

Moreover, feature monitoring integrates with the Hopsworks built-in system for alerts, enabling you to setup alerts that will notify you as soon as shift is detected in your feature values. You can setup alerts for feature monitoring at a Feature Group, Feature View, and project level.