com.logicalclocks.hsfs

Class FeatureViewBase<T extends FeatureViewBase,T3 extends FeatureStoreBase<T4>,T4 extends QueryBase,T5>

java.lang.Object

com.logicalclocks.hsfs.FeatureViewBase<T,T3,T4,T5>

Direct Known Subclasses:

FeatureView, FeatureView, FeatureView, FeatureView

public abstract class FeatureViewBase<T extends FeatureViewBase,T3 extends FeatureStoreBase<T4>,T4 extends QueryBase,T5>
extends Object

Field Summary

Fields

Modifier and Type	Field and Description
protected String	description
protected Integer	extraFilterVersion
protected List<TrainingDatasetFeature>	features
protected T3	featureStore
protected Integer	id
protected List<String>	labels
protected static org.slf4j.Logger	LOGGER
protected String	name
protected T4	query
protected String	type
protected VectorServer	vectorServer
protected Integer	version

Constructor Summary

Constructors

Constructor and Description
FeatureViewBase()

Method Summary

Modifier and Type	Method and Description
void	closeVectorServer() Closes the ExecutorService and JDBC DataSource used to retrieve feature vectors from the online feature store.
List<Object>	getFeatureVector(Map<String,Object> entry) Returns assembled feature vector from online feature store.
List<Object>	getFeatureVector(Map<String,Object> entry, boolean external) Returns assembled feature vector from online feature store.
List<List<Object>>	getFeatureVectors(Map<String,List<Object>> entry) Returns assembled feature vectors in batches from online feature store.
List<List<Object>>	getFeatureVectors(Map<String,List<Object>> entry, boolean external) Returns assembled feature vectors in batches from online feature store.
HashSet<String>	getPrimaryKeys() Set of primary key names that is used as keys in input dict object for `get_serving_vector` method.
void	initBatchScoring(Integer trainingDatasetVersion) Initialise feature view to retrieve feature vector from offline feature store.
void	initServing() Initialise feature view to retrieve feature vector from online feature store.
void	initServing(Boolean batch, Boolean external) Initialise feature view to retrieve feature vector from online feature store.
protected void	validateTrainTestSplit(Float testSize, String trainEnd, String testStart)
protected void	validateTrainValidationTestSplit(Float validationSize, Float testSize, String trainEnd, String validationStart, String validationEnd, String testStart)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

featureStore

protected T3 extends FeatureStoreBase<T4> featureStore

id

protected Integer id

name

protected String name

version

protected Integer version

description

protected String description

features

protected List<TrainingDatasetFeature> features

query

protected T4 extends QueryBase query

labels

protected List<String> labels

type

protected String type

LOGGER

protected static final org.slf4j.Logger LOGGER

vectorServer

protected VectorServer vectorServer

extraFilterVersion

protected Integer extraFilterVersion

Constructor Detail

FeatureViewBase

public FeatureViewBase()

Method Detail

initServing

public void initServing()
                 throws FeatureStoreException,
                        IOException,
                        SQLException,
                        ClassNotFoundException

Initialise feature view to retrieve feature vector from online feature store.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        // get feature view handle
        FeatureView fv = fs.getFeatureView("fv_name", 1);
        // Initialise feature view serving
        fv.initServing();
 
 

Throws:

FeatureStoreException - In case client is not connected to Hopsworks.

IOException - Generic IO exception.

SQLException - In case there is online storage (RonDB) access error or other errors.

ClassNotFoundException - In case class `com.mysql.jdbc.Driver` can not be found.

initServing

public void initServing(Boolean batch,
                        Boolean external)
                 throws FeatureStoreException,
                        IOException,
                        SQLException,
                        ClassNotFoundException

Initialise feature view to retrieve feature vector from online feature store.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        // get feature view handle
        FeatureView fv = fs.getFeatureView("fv_name", 1);
        // Initialise feature view batch serving
        fv.initServing(true, false);
 
 

Parameters:

batch - Whether to initialise feature view to retrieve feature vector from offline feature store.

external - If set to `true`, the connection to the online feature store is established using the same host as for the `host` parameter in the connection object. If set to False, the online feature store storage connector is used which relies on the private IP.

Throws:

FeatureStoreException - In case client is not connected to Hopsworks.

IOException - Generic IO exception.

SQLException - In case there is online storage (RonDB) access error or other errors.

ClassNotFoundException - In case class `com.mysql.jdbc.Driver` can not be found.

validateTrainTestSplit

protected void validateTrainTestSplit(Float testSize,
                                      String trainEnd,
                                      String testStart)
                               throws FeatureStoreException

Throws:

FeatureStoreException

validateTrainValidationTestSplit

protected void validateTrainValidationTestSplit(Float validationSize,
                                                Float testSize,
                                                String trainEnd,
                                                String validationStart,
                                                String validationEnd,
                                                String testStart)
                                         throws FeatureStoreException

Throws:

FeatureStoreException

initBatchScoring

public void initBatchScoring(Integer trainingDatasetVersion)

Initialise feature view to retrieve feature vector from offline feature store.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        // get feature view handle
        FeatureView fv = fs.getFeatureView("fv_name", 1);
        // Initialise feature view batch scoring
        fv.initBatchScoring(1);
 
 

Parameters:

trainingDatasetVersion - Version of training dataset to identify additional filters attached to the training dataset and statistics to use for transformation functions.

getFeatureVector

public List<Object> getFeatureVector(Map<String,Object> entry)
                              throws FeatureStoreException,
                                     IOException,
                                     ClassNotFoundException

Returns assembled feature vector from online feature store.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        // get feature view handle
        FeatureView fv = fs.getFeatureView("fv_name", 1);
        // define primary key values to fetch data from online feature store
        Map<String, Object> pkMap = new HashMap<String, Object>() {
               {put("customer_id", 1);
                put("contract_id" , 100);
                }
        };
        // get feature vector
        fv.getFeatureVector(entry);
 
 

Parameters:

entry - Fictionary of feature group primary key and values provided by serving application.

Returns:

List of feature values related to provided primary keys, ordered according to positions of the features in the feature view query.

Throws:

FeatureStoreException - In case client is not connected to Hopsworks.

IOException - Generic IO exception.

ClassNotFoundException - In case class `com.mysql.jdbc.Driver` can not be found.

getFeatureVector

public List<Object> getFeatureVector(Map<String,Object> entry,
                                     boolean external)
                              throws FeatureStoreException,
                                     IOException,
                                     ClassNotFoundException

Returns assembled feature vector from online feature store.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        // get feature view handle
        FeatureView fv = fs.getFeatureView("fv_name", 1);
        // define primary key values to fetch data from online feature store
        Map<String, Object> pkMap = new HashMap<String, Object>() {
               {put("customer_id", 1);
                put("contract_id" , 100);
                }
        };
        // get feature vector
        fv.getFeatureVector(entry, false);
 
 

Parameters:

entry - Dictionary of feature group primary key and values provided by serving application.

external - If set to true, the connection to the online feature store is established using the same host as for the `host` parameter in the connection object. If set to false, the online feature store storage connector is used which relies on the private IP. Defaults to True if connection to Hopsworks is established from external environment

Returns:

List of feature values related to provided primary keys, ordered according to positions of the features in the feature view query.

Throws:

FeatureStoreException - In case client is not connected to Hopsworks.

IOException - Generic IO exception.

ClassNotFoundException - In case class `com.mysql.jdbc.Driver` can not be found.

getFeatureVectors

public List<List<Object>> getFeatureVectors(Map<String,List<Object>> entry)
                                     throws SQLException,
                                            FeatureStoreException,
                                            IOException

Returns assembled feature vectors in batches from online feature store.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        // get feature view handle
        FeatureView fv = fs.getFeatureView("fv_name", 1);
        // define primary key values to fetch data from online feature store
        Map<String, List<Long>> entry = ...;
        // get feature vector
        fv.getFeatureVector(entry);
 
 

Parameters:

entry - A list of dictionaries of feature group primary key and values provided by serving application.

Returns:

List of lists of feature values related to provided primary keys, ordered according to positions of the features in the feature view query.

Throws:

FeatureStoreException - In case client is not connected to Hopsworks.

IOException - Generic IO exception.

SQLException - In case there is online storage (RonDB) access error or other errors.

getFeatureVectors

public List<List<Object>> getFeatureVectors(Map<String,List<Object>> entry,
                                            boolean external)
                                     throws SQLException,
                                            FeatureStoreException,
                                            IOException,
                                            ClassNotFoundException

Returns assembled feature vectors in batches from online feature store.

 
        // get feature store handle
        FeatureStore fs = HopsworksConnection.builder().build().getFeatureStore();
        // get feature view handle
        FeatureView fv = fs.getFeatureView("fv_name", 1);
        // define primary key values to fetch data from online feature store
        Map<String, List<Long>> entry = ...;
        // get feature vector
        fv.getFeatureVectors(entry, false);
 
 

Parameters:

entry - A list of dictionaries of feature group primary key and values provided by serving application.

external - If set to `true`, the connection to the online feature store is established using the same host as for the `host` parameter in the connection object. If set to False, the online feature store storage connector is used which relies on the private IP.

Returns:

List of lists of feature values related to provided primary keys, ordered according to positions of this features in the feature view query.

Throws:

FeatureStoreException - In case client is not connected to Hopsworks.

IOException - Generic IO exception.

SQLException - In case there is online storage (RonDB) access error or other errors.

ClassNotFoundException - In case class `com.mysql.jdbc.Driver` can not be found.

getPrimaryKeys

public HashSet<String> getPrimaryKeys()
                               throws SQLException,
                                      IOException,
                                      FeatureStoreException,
                                      ClassNotFoundException

Set of primary key names that is used as keys in input dict object for `get_serving_vector` method.

Returns:

Set of serving keys

Throws:

SQLException

IOException

FeatureStoreException

ClassNotFoundException

closeVectorServer

public void closeVectorServer()

Closes the ExecutorService and JDBC DataSource used to retrieve feature vectors from the online feature store.