feast package

Submodules

feast.cli module

feast.client module

class feast.client.Client(options: Optional[Dict[str, str]] = None, **kwargs)[source]

Bases: object

Feast Client: Used for creating, managing, and retrieving features.

apply(objects: Union[List[Union[feast.entity.Entity, feast.feature_table.FeatureTable]], feast.entity.Entity, feast.feature_table.FeatureTable], project: Optional[str] = None)[source]

Idempotently registers entities and feature tables with Feast Core. Either a single entity or feature table or a list can be provided.

Parameters

objects – List of entities and/or feature tables that will be registered

Examples

>>> from feast import Client
>>> from feast.entity import Entity
>>> from feast.value_type import ValueType
>>>
>>> feast_client = Client(core_url="localhost:6565")
>>> entity = Entity(
>>>     name="driver_entity",
>>>     description="Driver entity for car rides",
>>>     value_type=ValueType.STRING,
>>>     labels={
>>>         "key": "val"
>>>     }
>>> )
>>> feast_client.apply(entity)
apply_entity(entities: Union[List[feast.entity.Entity], feast.entity.Entity], project: Optional[str] = None)[source]

Deprecated. Please see apply().

apply_feature_table(feature_tables: Union[List[feast.feature_table.FeatureTable], feast.feature_table.FeatureTable], project: Optional[str] = None)[source]

Deprecated. Please see apply().

archive_project(project)[source]

Archives a project. Project will still continue to function for ingestion and retrieval, but will be in a read-only state. It will also not be visible from the Core API for management purposes.

Parameters

project – Name of project to archive

property config
property core_secure

Retrieve Feast Core client-side SSL/TLS setting

Returns

Whether client-side SSL/TLS is enabled

property core_url

Retrieve Feast Core URL

Returns

Feast Core URL string

create_project(project: str)[source]

Creates a Feast project

Parameters

project – Name of project

delete_feature_table(name: str, project: Optional[str] = None)None[source]

Deletes a feature table.

Parameters
  • project – Feast project that this feature table belongs to

  • name – Name of feature table

get_entity(name: str, project: Optional[str] = None)feast.entity.Entity[source]

Retrieves an entity.

Parameters
  • project – Feast project that this entity belongs to

  • name – Name of entity

Returns

Returns either the specified entity, or raises an exception if none is found

get_feature_table(name: str, project: Optional[str] = None)feast.feature_table.FeatureTable[source]

Retrieves a feature table.

Parameters
  • project – Feast project that this feature table belongs to

  • name – Name of feature table

Returns

Returns either the specified feature table, or raises an exception if none is found

get_online_features(feature_refs: List[str], entity_rows: List[Dict[str, Any]], project: Optional[str] = None)feast.online_response.OnlineResponse[source]

Retrieves the latest online feature data from Feast Serving. :param feature_refs: List of feature references that will be returned for each entity.

Each feature reference should have the following format: “feature_table:feature” where “feature_table” & “feature” refer to the feature and feature table names respectively. Only the feature name is required.

Parameters
  • entity_rows – A list of dictionaries where each key-value is an entity-name, entity-value pair.

  • project – Optionally specify the the project override. If specified, uses given project for retrieval. Overrides the projects specified in Feature References if also are specified.

Returns

GetOnlineFeaturesResponse containing the feature data in records. Each EntityRow provided will yield one record, which contains data fields with data value and field status metadata (if included).

Examples

>>> from feast import Client
>>>
>>> feast_client = Client(core_url="localhost:6565", serving_url="localhost:6566")
>>> feature_refs = ["sales:daily_transactions"]
>>> entity_rows = [{"customer_id": 0},{"customer_id": 1}]
>>>
>>> online_response = feast_client.get_online_features(
>>>     feature_refs, entity_rows, project="my_project")
>>> online_response_dict = online_response.to_dict()
>>> print(online_response_dict)
{'sales:daily_transactions': [1.1,1.2], 'sales:customer_id': [0,1]}
ingest(feature_table: Union[str, feast.feature_table.FeatureTable], source: Union[pandas.core.frame.DataFrame, str], project: Optional[str] = None, chunk_size: int = 10000, max_workers: int = 1, timeout: int = 120)None[source]

Batch load feature data into a FeatureTable.

Parameters
  • feature_table (typing.Union[str, feast.feature_table.FeatureTable]) – FeatureTable object or the string name of the feature table

  • source (typing.Union[pd.DataFrame, str]) –

    Either a file path or Pandas Dataframe to ingest into Feast Files that are currently supported:

    • parquet

    • csv

    • json

  • project – Feast project to locate FeatureTable

  • chunk_size (int) – Amount of rows to load and ingest at a time.

  • max_workers (int) – Number of worker processes to use to encode values.

  • timeout (int) – Timeout in seconds to wait for completion.

Examples

>>> from feast import Client
>>>
>>> client = Client(core_url="localhost:6565")
>>> ft_df = pd.DataFrame(
>>>         {
>>>            "datetime": [pd.datetime.now()],
>>>            "driver": [1001],
>>>            "rating": [4.3],
>>>         }
>>>     )
>>> client.set_project("project1")
>>>
>>> driver_ft = client.get_feature_table("driver")
>>> client.ingest(driver_ft, ft_df)
property job_service_secure

Retrieve Feast Job Service client-side SSL/TLS setting

Returns

Whether client-side SSL/TLS is enabled

property job_service_url

Retrieve Feast Job Service URL

Returns

Feast Job Service URL string

list_entities(project: Optional[str] = None, labels: Dict[str, str] = {})List[feast.entity.Entity][source]

Retrieve a list of entities from Feast Core

Parameters
  • project – Filter entities based on project name

  • labels – User-defined labels that these entities are associated with

Returns

List of entities

list_feature_tables(project: Optional[str] = None, labels: Dict[str, str] = {})List[feast.feature_table.FeatureTable][source]

Retrieve a list of feature tables from Feast Core

Parameters

project – Filter feature tables based on project name

Returns

List of feature tables

list_features_by_ref(project: Optional[str] = None, entities: List[str] = [], labels: Dict[str, str] = {})Dict[feast.feature.FeatureRef, feast.feature.Feature][source]

Retrieve a dictionary of feature reference to feature from Feast Core based on filters provided.

Parameters
  • project – Feast project that these features belongs to

  • entities – Feast entity that these features are associated with

  • labels – Feast labels that these features are associated with

Returns

features>

Return type

Dictionary of <feature references

Examples

>>> from feast import Client
>>>
>>> feast_client = Client(core_url="localhost:6565")
>>> features = feast_client.list_features(project="test_project", entities=["driver_id"], labels={"key1":"val1","key2":"val2"})
>>> print(features)
list_projects()List[str][source]

List all active Feast projects

Returns

List of project names

property project

Retrieve currently active project

Returns

Project name

property serving_secure

Retrieve Feast Serving client-side SSL/TLS setting

Returns

Whether client-side SSL/TLS is enabled

property serving_url

Retrieve Feast Serving URL

Returns

Feast Serving URL string

set_project(project: Optional[str] = None)[source]

Set currently active Feast project

Parameters

project – Project to set as active. If unset, will reset to the default project.

version(sdk_only=False)[source]

Returns version information from Feast Core and Feast Serving

feast.config module

class feast.config.Config(options: Optional[Dict[str, str]] = None, path: Optional[str] = None)[source]

Bases: object

Maintains and provides access to Feast configuration

Configuration is stored as key/value pairs. The user can specify options through either input arguments to this class, environmental variables, or by setting the config in a configuration file

exists(option)[source]

Tests whether a specific option is available

Parameters

option – Name of the option to check

Returns: Boolean true/false whether the option is set

get(option, default=<object object>)[source]

Returns a single configuration option as a string

Parameters
  • option – Name of the option

  • default – Default value to return if option is not found

Returns: String option that is returned

getboolean(option, default=<object object>)[source]

Returns a single configuration option as a boolean

Parameters
  • option – Name of the option

  • default – Default value to return if option is not found

Returns: Boolean option value that is returned

getfloat(option, default=<object object>)[source]

Returns a single configuration option as an integer

Parameters
  • option – Name of the option

  • default – Default value to return if option is not found

Returns: Float option value that is returned

getint(option, default=<object object>)[source]

Returns a single configuration option as an integer

Parameters
  • option – Name of the option

  • default – Default value to return if option is not found

Returns: Integer option value that is returned

save()[source]

Save the current configuration to disk. This does not include environmental variables or initialized options

set(option, value)[source]

Sets a configuration option. Must be serializable to string :param option: Option name to use as key :param value: Value to store under option

feast.constants module

class feast.constants.AuthProvider(value)[source]

Bases: enum.Enum

An enumeration.

GOOGLE = 'google'
OAUTH = 'oauth'
feast.constants.CONFIG_FEAST_ENV_VAR_PREFIX: str = 'FEAST_'

Default prefix to Feast environmental variables

feast.constants.CONFIG_FILE_DEFAULT_DIRECTORY: str = '.feast'

Default directory to Feast configuration file

feast.constants.CONFIG_FILE_NAME: str = 'config'

Default Feast configuration file name

feast.constants.CONFIG_FILE_SECTION: str = 'general'

Default section in Feast configuration file to specify options

class feast.constants.ConfigMeta(name, bases, attrs)[source]

Bases: type

Class factory which customizes ConfigOptions class instantiation. Specifically, setting configuration option’s name to lowercase of capitalized variable.

class feast.constants.ConfigOptions[source]

Bases: object

Feast Configuration Options

AUTH_PROVIDER: str = 'auth_provider'

Authentication Provider - Google OpenID/OAuth

Options: “google” / “oauth”

AUTH_TOKEN: Optional[str] = 'auth_token'

JWT Auth token for user authentication to Feast

AZURE_BLOB_ACCOUNT_ACCESS_KEY: Optional[str] = 'azure_blob_account_access_key'

Account access key for Azure blob storage_client

AZURE_BLOB_ACCOUNT_NAME: Optional[str] = 'azure_blob_account_name'

Account name for Azure blob storage_client

BATCH_FEATURE_REQUEST_WAIT_TIME_SECONDS: str = 'batch_feature_request_wait_time_seconds'

Time to wait for historical feature requests before timing out.

BATCH_INGESTION_PRODUCTION_TIMEOUT: str = 'batch_ingestion_production_timeout'

Default timeout when running batch ingestion

CORE_ENABLE_SSL: str = 'core_enable_ssl'

Enable or disable TLS/SSL to Feast Core

CORE_SERVER_SSL_CERT: str = 'core_server_ssl_cert'

Path to certificate(s) to secure connection to Feast Core

CORE_URL: str = 'core_url'

Default Feast Core URL

DATAPROC_CLUSTER_NAME: Optional[str] = 'dataproc_cluster_name'

Dataproc cluster to run Feast Spark Jobs in

DATAPROC_EXECUTOR_CORES = 'dataproc_executor_cores'

No. of executor cores for Dataproc cluster

DATAPROC_EXECUTOR_INSTANCES = 'dataproc_executor_instances'

No. of executor instances for Dataproc cluster

DATAPROC_EXECUTOR_MEMORY = 'dataproc_executor_memory'

No. of executor memory for Dataproc cluster

DATAPROC_PROJECT: Optional[str] = 'dataproc_project'

Project of Dataproc cluster

DATAPROC_REGION: Optional[str] = 'dataproc_region'

Region of Dataproc cluster

DEADLETTER_PATH: str = 'deadletter_path'

Ingestion Job DeadLetter Destination. The choice of storage is connected to the choice of SPARK_LAUNCHER.

Eg. gs://some-bucket/output/, s3://some-bucket/output/, file:///data/subfolder/

EMR_CLUSTER_ID: Optional[str] = 'emr_cluster_id'

EMR cluster to run Feast Spark Jobs in

EMR_CLUSTER_TEMPLATE_PATH: Optional[str] = 'emr_cluster_template_path'

Template path of EMR cluster

EMR_LOG_LOCATION: Optional[str] = 'emr_log_location'

Log path of EMR cluster

EMR_REGION: Optional[str] = 'emr_region'

Region of EMR cluster

ENABLE_AUTH: str = 'enable_auth'

Enable user authentication to Feast Core

GRPC_CONNECTION_TIMEOUT: str = 'grpc_connection_timeout'

Default connection timeout to Feast Serving, Feast Core, and Feast Job Service (in seconds)

GRPC_CONNECTION_TIMEOUT_APPLY: str = 'grpc_connection_timeout_apply'

Default gRPC connection timeout when sending an ApplyFeatureTable command to Feast Core (in seconds)

HISTORICAL_FEATURE_OUTPUT_FORMAT: str = 'historical_feature_output_format'

File format of historical retrieval features

HISTORICAL_FEATURE_OUTPUT_LOCATION: Optional[str] = 'historical_feature_output_location'

File location of historical retrieval features

INGESTION_DROP_INVALID_ROWS = 'ingestion_drop_invalid_rows'

If set to true rows that do not pass custom validation (see feast.contrib.validation) won’t be saved to Online Storage

JOB_SERVICE_ENABLE_CONTROL_LOOP: str = 'job_service_enable_control_loop'

Enable or disable control loop for Feast Job Service

JOB_SERVICE_ENABLE_SSL: str = 'job_service_enable_ssl'

Enable or disable TLS/SSL to Feast Job Service

JOB_SERVICE_SERVER_SSL_CERT: str = 'job_service_server_ssl_cert'

Path to certificate(s) to secure connection to Feast Job Service

JOB_SERVICE_URL: Optional[str] = 'job_service_url'

Default Feast Job Service URL

OAUTH_AUDIENCE: Optional[str] = 'oauth_audience'

Oauth intended recipients

OAUTH_CLIENT_ID: Optional[str] = 'oauth_client_id'

Oauth client ID

OAUTH_CLIENT_SECRET: Optional[str] = 'oauth_client_secret'

Oauth client secret

OAUTH_GRANT_TYPE: Optional[str] = 'oauth_grant_type'

Oauth grant type

OAUTH_TOKEN_REQUEST_URL: Optional[str] = 'oauth_token_request_url'

Oauth token request url

PROJECT: str = 'project'

Feast project namespace to use

REDIS_HOST: str = 'redis_host'

Default Redis host

REDIS_PORT: str = 'redis_port'

Default Redis port

REDIS_SSL: str = 'redis_ssl'

Enable or disable TLS/SSL to Redis

REGISTRY_PATH: Optional[str] = 'registry_path'

Object store registry

S3_ENDPOINT_URL: Optional[str] = 's3_endpoint_url'

Endpoint URL for S3 storage_client

SERVING_ENABLE_SSL: str = 'serving_enable_ssl'

Enable or disable TLS/SSL to Feast Serving

SERVING_SERVER_SSL_CERT: str = 'serving_server_ssl_cert'

Path to certificate(s) to secure connection to Feast Serving

SERVING_URL: str = 'serving_url'

Default Feast Serving URL

SPARK_BQ_MATERIALIZATION_DATASET: Optional[str] = 'spark_bq_materialization_dataset'

The dataset id where the materialized view of BigQuerySource is going to be created by default, use the same dataset where view is located

SPARK_BQ_MATERIALIZATION_PROJECT: Optional[str] = 'spark_bq_materialization_project'

The project id where the materialized view of BigQuerySource is going to be created by default, use the same project where view is located

SPARK_HOME: Optional[str] = 'spark_home'

Directory where Spark is installed

SPARK_INGESTION_JAR: str = 'spark_ingestion_jar'

Feast Spark Job ingestion jar file. The choice of storage is connected to the choice of SPARK_LAUNCHER.

Eg. “dataproc” (http and gs), “emr” (http and s3), “standalone” (http and file)

SPARK_K8S_JOB_TEMPLATE_PATH = 'spark_k8s_job_template_path'
SPARK_K8S_NAMESPACE = 'spark_k8s_namespace'
SPARK_K8S_USE_INCLUSTER_CONFIG = 'spark_k8s_use_incluster_config'
SPARK_LAUNCHER: Optional[str] = 'spark_launcher'

Spark Job launcher. The choice of storage is connected to the choice of SPARK_LAUNCHER.

Options: “standalone”, “dataproc”, “emr”

SPARK_STAGING_LOCATION: Optional[str] = 'spark_staging_location'

Feast Spark Job ingestion jobs staging location. The choice of storage is connected to the choice of SPARK_LAUNCHER.

Eg. gs://some-bucket/output/, s3://some-bucket/output/, file:///data/subfolder/

SPARK_STANDALONE_MASTER: str = 'spark_standalone_master'

Spark resource manager master url

STATSD_ENABLED: str = 'statsd_enabled'

Enable or disable StatsD

STATSD_HOST: Optional[str] = 'statsd_host'

Default StatsD port

STATSD_PORT: Optional[str] = 'statsd_port'

Default StatsD port

STENCIL_URL: str = 'stencil_url'

ProtoRegistry Address (currently only Stencil Server is supported as registry) https://github.com/gojekfarm/stencil

TELEMETRY = 'telemetry'

Telemetry enabled

defaults()[source]
feast.constants.DATETIME_COLUMN: str = 'datetime'

Default datetime column name for point-in-time join

feast.constants.FEAST_CONFIG_FILE_ENV: str = 'FEAST_CONFIG'

Environmental variable to specify Feast configuration file location

class feast.constants.Option(name, default)[source]

Bases: object

feast.data_format module

class feast.data_format.AvroFormat(schema_json: str)[source]

Bases: feast.data_format.StreamFormat

Defines the Avro streaming data format that encodes data in Avro format

to_proto()[source]

Convert this StreamFormat into its protobuf representation.

class feast.data_format.FileFormat[source]

Bases: abc.ABC

Defines an abtract file forma used to encode feature data in files

classmethod from_proto(proto)[source]

Construct this FileFormat from its protobuf representation. Raises NotImplementedError if FileFormat specified in given proto is not supported.

abstract to_proto()[source]

Convert this FileFormat into its protobuf representation.

class feast.data_format.ParquetFormat[source]

Bases: feast.data_format.FileFormat

Defines the Parquet data format

to_proto()[source]

Convert this FileFormat into its protobuf representation.

class feast.data_format.ProtoFormat(class_path: str)[source]

Bases: feast.data_format.StreamFormat

Defines the Protobuf data format

to_proto()[source]

Convert this StreamFormat into its protobuf representation.

class feast.data_format.StreamFormat[source]

Bases: abc.ABC

Defines an abtracts streaming data format used to encode feature data in streams

classmethod from_proto(proto)[source]

Construct this StreamFormat from its protobuf representation.

abstract to_proto()[source]

Convert this StreamFormat into its protobuf representation.

feast.data_source module

class feast.data_source.BigQueryOptions(table_ref: Optional[str], query: Optional[str])[source]

Bases: object

DataSource BigQuery options used to source features from BigQuery query

classmethod from_proto(bigquery_options_proto: feast.core.DataSource_pb2.BigQueryOptions)[source]

Creates a BigQueryOptions from a protobuf representation of a BigQuery option

Parameters

bigquery_options_proto – A protobuf representation of a DataSource

Returns

Returns a BigQueryOptions object based on the bigquery_options protobuf

property query

Returns the BigQuery SQL query referenced by this source

property table_ref

Returns the table ref of this BQ table

to_proto()feast.core.DataSource_pb2.BigQueryOptions[source]

Converts an BigQueryOptionsProto object to its protobuf representation.

Returns

BigQueryOptionsProto protobuf

class feast.data_source.BigQuerySource(event_timestamp_column: str, table_ref: Optional[str] = None, created_timestamp_column: Optional[str] = '', field_mapping: Optional[Dict[str, str]] = None, date_partition_column: Optional[str] = '', query: Optional[str] = None)[source]

Bases: feast.data_source.DataSource

property bigquery_options

Returns the bigquery options of this data source

get_table_query_string()str[source]

Returns a string that can directly be used to reference this table in SQL

property query
property table_ref
to_proto()feast.core.DataSource_pb2.DataSource[source]

Converts an DataSourceProto object to its protobuf representation.

class feast.data_source.DataSource(event_timestamp_column: str, created_timestamp_column: Optional[str] = '', field_mapping: Optional[Dict[str, str]] = None, date_partition_column: Optional[str] = '')[source]

Bases: object

DataSource that can be used source features

property created_timestamp_column

Returns the created timestamp column of this data source

property date_partition_column

Returns the date partition column of this data source

property event_timestamp_column

Returns the event timestamp column of this data source

property field_mapping

Returns the field mapping of this data source

static from_proto(data_source)[source]

Convert data source config in FeatureTable spec to a DataSource class object.

to_proto()feast.core.DataSource_pb2.DataSource[source]

Converts an DataSourceProto object to its protobuf representation.

class feast.data_source.FileOptions(file_format: Optional[feast.data_format.FileFormat], file_url: Optional[str])[source]

Bases: object

DataSource File options used to source features from a file

property file_format

Returns the file format of this file

property file_url

Returns the file url of this file

classmethod from_proto(file_options_proto: feast.core.DataSource_pb2.FileOptions)[source]

Creates a FileOptions from a protobuf representation of a file option

Parameters

file_options_proto – a protobuf representation of a datasource

Returns

Returns a FileOptions object based on the file_options protobuf

to_proto()feast.core.DataSource_pb2.FileOptions[source]

Converts an FileOptionsProto object to its protobuf representation.

Returns

FileOptionsProto protobuf

class feast.data_source.FileSource(event_timestamp_column: str, file_url: Optional[str] = None, path: Optional[str] = None, file_format: Optional[feast.data_format.FileFormat] = None, created_timestamp_column: Optional[str] = '', field_mapping: Optional[Dict[str, str]] = None, date_partition_column: Optional[str] = '')[source]

Bases: feast.data_source.DataSource

property file_options

Returns the file options of this data source

property path

Returns the file path of this feature data source

to_proto()feast.core.DataSource_pb2.DataSource[source]

Converts an DataSourceProto object to its protobuf representation.

class feast.data_source.KafkaOptions(bootstrap_servers: str, message_format: feast.data_format.StreamFormat, topic: str)[source]

Bases: object

DataSource Kafka options used to source features from Kafka messages

property bootstrap_servers

Returns a comma-separated list of Kafka bootstrap servers

classmethod from_proto(kafka_options_proto: feast.core.DataSource_pb2.KafkaOptions)[source]

Creates a KafkaOptions from a protobuf representation of a kafka option

Parameters

kafka_options_proto – A protobuf representation of a DataSource

Returns

Returns a BigQueryOptions object based on the kafka_options protobuf

property message_format

Returns the data format that is used to encode the feature data in Kafka messages

to_proto()feast.core.DataSource_pb2.KafkaOptions[source]

Converts an KafkaOptionsProto object to its protobuf representation.

Returns

KafkaOptionsProto protobuf

property topic

Returns the Kafka topic to collect feature data from

class feast.data_source.KafkaSource(event_timestamp_column: str, bootstrap_servers: str, message_format: feast.data_format.StreamFormat, topic: str, created_timestamp_column: Optional[str] = '', field_mapping: Optional[Dict[str, str]] = {}, date_partition_column: Optional[str] = '')[source]

Bases: feast.data_source.DataSource

property kafka_options

Returns the kafka options of this data source

to_proto()feast.core.DataSource_pb2.DataSource[source]

Converts an DataSourceProto object to its protobuf representation.

class feast.data_source.KinesisOptions(record_format: feast.data_format.StreamFormat, region: str, stream_name: str)[source]

Bases: object

DataSource Kinesis options used to source features from Kinesis records

classmethod from_proto(kinesis_options_proto: feast.core.DataSource_pb2.KinesisOptions)[source]

Creates a KinesisOptions from a protobuf representation of a kinesis option

Parameters

kinesis_options_proto – A protobuf representation of a DataSource

Returns

Returns a KinesisOptions object based on the kinesis_options protobuf

property record_format

Returns the data format used to encode the feature data in the Kinesis records.

property region

Returns the AWS region of Kinesis stream

property stream_name

Returns the Kinesis stream name to obtain feature data from

to_proto()feast.core.DataSource_pb2.KinesisOptions[source]

Converts an KinesisOptionsProto object to its protobuf representation.

Returns

KinesisOptionsProto protobuf

class feast.data_source.KinesisSource(event_timestamp_column: str, created_timestamp_column: str, record_format: feast.data_format.StreamFormat, region: str, stream_name: str, field_mapping: Optional[Dict[str, str]] = {}, date_partition_column: Optional[str] = '')[source]

Bases: feast.data_source.DataSource

property kinesis_options

Returns the kinesis options of this data source

to_proto()feast.core.DataSource_pb2.DataSource[source]

Converts an DataSourceProto object to its protobuf representation.

class feast.data_source.SourceType(value)[source]

Bases: enum.Enum

DataSource value type. Used to define source types in DataSource.

BATCH_BIGQUERY = 2
BATCH_FILE = 1
STREAM_KAFKA = 3
STREAM_KINESIS = 4
UNKNOWN = 0

feast.driver_test_data module

class feast.driver_test_data.EventTimestampType(value)[source]

Bases: enum.Enum

An enumeration.

TZ_AWARE_FIXED_OFFSET = 2
TZ_AWARE_US_PACIFIC = 3
TZ_AWARE_UTC = 1
TZ_NAIVE = 0
feast.driver_test_data.create_customer_daily_profile_df(customers, start_date, end_date)pandas.core.frame.DataFrame[source]

Example df generated by this function:

datetime | customer_id | current_balance | avg_passenger_count | lifetime_trip_count | created |

|------------------+-------------+-----------------+---------------------+---------------------+------------------| | 2021-03-17 19:31 | 1010 | 0.889188 | 0.049057 | 412 | 2021-03-24 19:38 | | 2021-03-18 19:31 | 1010 | 0.979273 | 0.212630 | 639 | 2021-03-24 19:38 | | 2021-03-19 19:31 | 1010 | 0.976549 | 0.176881 | 70 | 2021-03-24 19:38 | | 2021-03-20 19:31 | 1010 | 0.273697 | 0.325012 | 68 | 2021-03-24 19:38 | | 2021-03-21 19:31 | 1010 | 0.438262 | 0.313009 | 192 | 2021-03-24 19:38 | | | … | … | … | … | | | 2021-03-19 19:31 | 1001 | 0.738860 | 0.857422 | 344 | 2021-03-24 19:38 | | 2021-03-20 19:31 | 1001 | 0.848397 | 0.745989 | 106 | 2021-03-24 19:38 | | 2021-03-21 19:31 | 1001 | 0.301552 | 0.185873 | 812 | 2021-03-24 19:38 | | 2021-03-22 19:31 | 1001 | 0.943030 | 0.561219 | 322 | 2021-03-24 19:38 | | 2021-03-23 19:31 | 1001 | 0.354919 | 0.810093 | 273 | 2021-03-24 19:38 |

feast.driver_test_data.create_driver_hourly_stats_df(drivers, start_date, end_date)pandas.core.frame.DataFrame[source]

Example df generated by this function:

datetime | driver_id | conv_rate | acc_rate | avg_daily_trips | created |

|------------------+-----------+-----------+----------+-----------------+------------------| | 2021-03-17 19:31 | 5010 | 0.229297 | 0.685843 | 861 | 2021-03-24 19:34 | | 2021-03-17 20:31 | 5010 | 0.781655 | 0.861280 | 769 | 2021-03-24 19:34 | | 2021-03-17 21:31 | 5010 | 0.150333 | 0.525581 | 778 | 2021-03-24 19:34 | | 2021-03-17 22:31 | 5010 | 0.951701 | 0.228883 | 570 | 2021-03-24 19:34 | | 2021-03-17 23:31 | 5010 | 0.819598 | 0.262503 | 473 | 2021-03-24 19:34 | | | … | … | … | … | | | 2021-03-24 16:31 | 5001 | 0.061585 | 0.658140 | 477 | 2021-03-24 19:34 | | 2021-03-24 17:31 | 5001 | 0.088949 | 0.303897 | 618 | 2021-03-24 19:34 | | 2021-03-24 18:31 | 5001 | 0.096652 | 0.747421 | 480 | 2021-03-24 19:34 | | 2021-03-17 19:31 | 5005 | 0.142936 | 0.707596 | 466 | 2021-03-24 19:34 | | 2021-03-17 19:31 | 5005 | 0.142936 | 0.707596 | 466 | 2021-03-24 19:34 |

feast.driver_test_data.create_orders_df(customers, drivers, start_date, end_date, order_count)pandas.core.frame.DataFrame[source]

Example df generated by this function:

order_id | driver_id | customer_id | order_is_success | event_timestamp |

feast.entity module

class feast.entity.Entity(name: str, value_type: feast.value_type.ValueType, description: str = '', join_key: Optional[str] = None, labels: Optional[MutableMapping[str, str]] = None)[source]

Bases: object

Represents a collection of entities and associated metadata.

property created_timestamp

Returns the created_timestamp of this entity

property description

Returns the description of this entity

classmethod from_dict(entity_dict)[source]

Creates an entity from a dict

Parameters

entity_dict – A dict representation of an entity

Returns

Returns a EntityV2 object based on the entity dict

classmethod from_proto(entity_proto: feast.core.Entity_pb2.Entity)[source]

Creates an entity from a protobuf representation of an entity

Parameters

entity_proto – A protobuf representation of an entity

Returns

Returns a EntityV2 object based on the entity protobuf

classmethod from_yaml(yml: str)[source]

Creates an entity from a YAML string body or a file path

Parameters

yml – Either a file path containing a yaml file or a YAML string

Returns

Returns a EntityV2 object based on the YAML file

is_valid()[source]

Validates the state of a entity locally. Raises an exception if entity is invalid.

property join_key

Returns the join key of this entity

property labels

Returns the labels of this entity. This is the user defined metadata defined as a dictionary.

property last_updated_timestamp

Returns the last_updated_timestamp of this entity

property name

Returns the name of this entity

to_dict()Dict[source]

Converts entity to dict

Returns

Dictionary object representation of entity

to_proto()feast.core.Entity_pb2.Entity[source]

Converts an entity object to its protobuf representation

Returns

EntityV2Proto protobuf

to_spec_proto()feast.core.Entity_pb2.EntitySpecV2[source]

Converts an EntityV2 object to its protobuf representation. Used when passing EntitySpecV2 object to Feast request.

Returns

EntitySpecV2 protobuf

to_yaml()[source]

Converts a entity to a YAML string.

Returns

Entity string returned in YAML format

property value_type

Returns the type of this entity

feast.example_repo module

feast.feature module

class feast.feature.Feature(name: str, dtype: feast.value_type.ValueType, labels: Optional[MutableMapping[str, str]] = None)[source]

Bases: object

Feature field type

property dtype

Getter for data type of this field

classmethod from_proto(feature_proto: feast.core.Feature_pb2.FeatureSpecV2)[source]
Parameters

feature_proto – FeatureSpecV2 protobuf object

Returns

Feature object

property labels

Getter for labels of this field

property name

Getter for name of this field

to_proto()feast.core.Feature_pb2.FeatureSpecV2[source]

Converts Feature object to its Protocol Buffer representation

class feast.feature.FeatureRef(name: str, feature_table: str)[source]

Bases: object

Feature Reference represents a reference to a specific feature.

classmethod from_proto(proto: feast.serving.ServingService_pb2.FeatureReferenceV2)[source]

Construct a feature reference from the given FeatureReference proto Arg:

proto: Protobuf FeatureReference to construct from

Returns

FeatureRef that refers to the given feature

classmethod from_str(feature_ref_str: str)[source]

Parse the given string feature reference into FeatureRef model String feature reference should be in the format feature_table:feature. Where “feature_table” and “name” are the feature_table name and feature name respectively. :param feature_ref_str: String representation of the feature reference

Returns

FeatureRef that refers to the given feature

to_proto()feast.serving.ServingService_pb2.FeatureReferenceV2[source]

Convert and return this feature table reference to protobuf. :returns: Protobuf respresentation of this feature table reference.

feast.feature_store module

class feast.feature_store.FeatureStore(repo_path: Optional[str] = None, config: Optional[feast.repo_config.RepoConfig] = None)[source]

Bases: object

A FeatureStore object is used to define, create, and retrieve features.

apply(objects: Union[feast.entity.Entity, feast.feature_view.FeatureView, List[Union[feast.feature_view.FeatureView, feast.entity.Entity]]])[source]

Register objects to metadata store and update related infrastructure.

The apply method registers one or more definitions (e.g., Entity, FeatureView) and registers or updates these objects in the Feast registry. Once the registry has been updated, the apply method will update related infrastructure (e.g., create tables in an online store) in order to reflect these new definitions. All operations are idempotent, meaning they can safely be rerun.

Args: objects (List[Union[FeatureView, Entity]]): A list of FeatureView or Entity objects that should be

registered

Examples

Register a single Entity and FeatureView. >>> from feast.feature_store import FeatureStore >>> from feast import Entity, FeatureView, Feature, ValueType, FileSource >>> from datetime import timedelta >>> >>> fs = FeatureStore() >>> customer_entity = Entity(name=”customer”, value_type=ValueType.INT64, description=”customer entity”) >>> customer_feature_view = FeatureView( >>> name=”customer_fv”, >>> entities=[“customer”], >>> features=[Feature(name=”age”, dtype=ValueType.INT64)], >>> input=FileSource(path=”file.parquet”, event_timestamp_column=”timestamp”), >>> ttl=timedelta(days=1) >>> ) >>> fs.apply([customer_entity, customer_feature_view])

config: feast.repo_config.RepoConfig
delete_feature_view(name: str)[source]

Deletes a feature view or raises an exception if not found.

Parameters

name – Name of feature view

get_entity(name: str)feast.entity.Entity[source]

Retrieves an entity.

Parameters

name – Name of entity

Returns

Returns either the specified entity, or raises an exception if none is found

get_feature_view(name: str)feast.feature_view.FeatureView[source]

Retrieves a feature view.

Parameters

name – Name of feature view

Returns

Returns either the specified feature view, or raises an exception if none is found

get_historical_features(entity_df: Union[pandas.core.frame.DataFrame, str], feature_refs: List[str])feast.infra.offline_stores.offline_store.RetrievalJob[source]

Enrich an entity dataframe with historical feature values for either training or batch scoring.

This method joins historical feature data from one or more feature views to an entity dataframe by using a time travel join.

Each feature view is joined to the entity dataframe using all entities configured for the respective feature view. All configured entities must be available in the entity dataframe. Therefore, the entity dataframe must contain all entities found in all feature views, but the individual feature views can have different entities.

Time travel is based on the configured TTL for each feature view. A shorter TTL will limit the amount of scanning that will be done in order to find feature data for a specific entity key. Setting a short TTL may result in null values being returned.

Parameters
  • entity_df (Union[pd.DataFrame, str]) – An entity dataframe is a collection of rows containing all entity columns (e.g., customer_id, driver_id) on which features need to be joined, as well as a event_timestamp column used to ensure point-in-time correctness. Either a Pandas DataFrame can be provided or a string SQL query. The query must be of a format supported by the configured offline store (e.g., BigQuery)

  • feature_refs – A list of features that should be retrieved from the offline store. Feature references are of the format “feature_view:feature”, e.g., “customer_fv:daily_transactions”.

Returns

RetrievalJob which can be used to materialize the results.

Examples

Retrieve historical features using a BigQuery SQL entity dataframe >>> from feast.feature_store import FeatureStore >>> >>> fs = FeatureStore(config=RepoConfig(provider=”gcp”)) >>> retrieval_job = fs.get_historical_features( >>> entity_df=”SELECT event_timestamp, order_id, customer_id from gcp_project.my_ds.customer_orders”, >>> feature_refs=[“customer:age”, “customer:avg_orders_1d”, “customer:avg_orders_7d”] >>> ) >>> feature_data = job.to_df() >>> model.fit(feature_data) # insert your modeling framework here.

get_online_features(feature_refs: List[str], entity_rows: List[Dict[str, Any]])feast.online_response.OnlineResponse[source]

Retrieves the latest online feature data.

Note: This method will download the full feature registry the first time it is run. If you are using a remote registry like GCS or S3 then that may take a few seconds. The registry remains cached up to a TTL duration (which can be set to infinitey). If the cached registry is stale (more time than the TTL has passed), then a new registry will be downloaded synchronously by this method. This download may introduce latency to online feature retrieval. In order to avoid synchronous downloads, please call refresh_registry() prior to the TTL being reached. Remember it is possible to set the cache TTL to infinity (cache forever).

Parameters
  • feature_refs – List of feature references that will be returned for each entity. Each feature reference should have the following format: “feature_table:feature” where “feature_table” & “feature” refer to the feature and feature table names respectively. Only the feature name is required.

  • entity_rows – A list of dictionaries where each key-value is an entity-name, entity-value pair.

Returns

OnlineResponse containing the feature data in records.

Examples

>>> from feast import FeatureStore
>>>
>>> store = FeatureStore(repo_path="...")
>>> feature_refs = ["sales:daily_transactions"]
>>> entity_rows = [{"customer_id": 0},{"customer_id": 1}]
>>>
>>> online_response = store.get_online_features(
>>>     feature_refs, entity_rows, project="my_project")
>>> online_response_dict = online_response.to_dict()
>>> print(online_response_dict)
{'sales:daily_transactions': [1.1,1.2], 'sales:customer_id': [0,1]}
list_entities(allow_cache: bool = False)List[feast.entity.Entity][source]

Retrieve a list of entities from the registry

Parameters

allow_cache (bool) – Whether to allow returning entities from a cached registry

Returns

List of entities

list_feature_views()List[feast.feature_view.FeatureView][source]

Retrieve a list of feature views from the registry

Returns

List of feature views

materialize(start_date: datetime.datetime, end_date: datetime.datetime, feature_views: Optional[List[str]] = None)None[source]

Materialize data from the offline store into the online store.

This method loads feature data in the specified interval from either the specified feature views, or all feature views if none are specified, into the online store where it is available for online serving.

Parameters
  • start_date (datetime) – Start date for time range of data to materialize into the online store

  • end_date (datetime) – End date for time range of data to materialize into the online store

  • feature_views (List[str]) – Optional list of feature view names. If selected, will only run materialization for the specified feature views.

Examples

Materialize all features into the online store over the interval from 3 hours ago to 10 minutes ago. >>> from datetime import datetime, timedelta >>> from feast.feature_store import FeatureStore >>> >>> fs = FeatureStore(config=RepoConfig(provider=”gcp”)) >>> fs.materialize( >>> start_date=datetime.utcnow() - timedelta(hours=3), end_date=datetime.utcnow() - timedelta(minutes=10) >>> )

materialize_incremental(end_date: datetime.datetime, feature_views: Optional[List[str]] = None)None[source]

Materialize incremental new data from the offline store into the online store.

This method loads incremental new feature data up to the specified end time from either the specified feature views, or all feature views if none are specified, into the online store where it is available for online serving. The start time of the interval materialized is either the most recent end time of a prior materialization or (now - ttl) if no such prior materialization exists.

Parameters
  • end_date (datetime) – End date for time range of data to materialize into the online store

  • feature_views (List[str]) – Optional list of feature view names. If selected, will only run materialization for the specified feature views.

Examples

Materialize all features into the online store up to 5 minutes ago. >>> from datetime import datetime, timedelta >>> from feast.feature_store import FeatureStore >>> >>> fs = FeatureStore(config=RepoConfig(provider=”gcp”, registry=”gs://my-fs/”, project=”my_fs_proj”)) >>> fs.materialize_incremental(end_date=datetime.utcnow() - timedelta(minutes=5))

property project
refresh_registry()[source]

Fetches and caches a copy of the feature registry in memory.

Explicitly calling this method allows for direct control of the state of the registry cache. Every time this method is called the complete registry state will be retrieved from the remote registry store backend (e.g., GCS, S3), and the cache timer will be reset. If refresh_registry() is run before get_online_features() is called, then get_online_feature() will use the cached registry instead of retrieving (and caching) the registry itself.

Additionally, the TTL for the registry cache can be set to infinity (by setting it to 0), which means that refresh_registry() will become the only way to update the cached registry. If the TTL is set to a value greater than 0, then once the cache becomes stale (more time than the TTL has passed), a new cache will be downloaded synchronously, which may increase latencies if the triggering method is get_online_features()

repo_path: pathlib.Path
version()str[source]

Returns the version of the current Feast SDK/CLI

feast.feature_table module

class feast.feature_table.FeatureTable(name: str, entities: List[str], features: List[feast.feature.Feature], batch_source: Optional[Union[feast.data_source.BigQuerySource, feast.data_source.FileSource]] = None, stream_source: Optional[Union[feast.data_source.KafkaSource, feast.data_source.KinesisSource]] = None, max_age: Optional[google.protobuf.duration_pb2.Duration] = None, labels: Optional[MutableMapping[str, str]] = None)[source]

Bases: object

Represents a collection of features and associated metadata.

add_feature(feature: feast.feature.Feature)[source]

Adds a new feature to the feature table.

property batch_source

Returns the batch source of this feature table

property created_timestamp

Returns the created_timestamp of this feature table

property entities

Returns the entities of this feature table

property features

Returns the features of this feature table

classmethod from_dict(ft_dict)[source]

Creates a feature table from a dict

Parameters

ft_dict – A dict representation of a feature table

Returns

Returns a FeatureTable object based on the feature table dict

classmethod from_proto(feature_table_proto: feast.core.FeatureTable_pb2.FeatureTable)[source]

Creates a feature table from a protobuf representation of a feature table

Parameters

feature_table_proto – A protobuf representation of a feature table

Returns

Returns a FeatureTableProto object based on the feature table protobuf

classmethod from_yaml(yml: str)[source]

Creates a feature table from a YAML string body or a file path

Parameters

yml – Either a file path containing a yaml file or a YAML string

Returns

Returns a FeatureTable object based on the YAML file

is_valid()[source]

Validates the state of a feature table locally. Raises an exception if feature table is invalid.

property labels

Returns the labels of this feature table. This is the user defined metadata defined as a dictionary.

property last_updated_timestamp

Returns the last_updated_timestamp of this feature table

property max_age

Returns the maximum age of this feature table. This is the total maximum amount of staleness that will be allowed during feature retrieval for each specific feature that is looked up.

property name

Returns the name of this feature table

property stream_source

Returns the stream source of this feature table

to_dict()Dict[source]

Converts feature table to dict

Returns

Dictionary object representation of feature table

to_proto()feast.core.FeatureTable_pb2.FeatureTable[source]

Converts an feature table object to its protobuf representation

Returns

FeatureTableProto protobuf

to_spec_proto()feast.core.FeatureTable_pb2.FeatureTableSpec[source]

Converts an FeatureTableProto object to its protobuf representation. Used when passing FeatureTableSpecProto object to Feast request.

Returns

FeatureTableSpecProto protobuf

to_yaml()[source]

Converts a feature table to a YAML string.

Returns

Feature table string returned in YAML format

feast.feature_view module

class feast.feature_view.FeatureView(name: str, entities: List[str], features: List[feast.feature.Feature], ttl: Optional[Union[google.protobuf.duration_pb2.Duration, datetime.timedelta]], input: Union[feast.data_source.BigQuerySource, feast.data_source.FileSource], tags: Optional[Dict[str, str]] = None, online: bool = True)[source]

Bases: object

A FeatureView defines a logical grouping of serveable features.

created_timestamp: Optional[google.protobuf.timestamp_pb2.Timestamp] = None
entities: List[str]
features: List[feast.feature.Feature]
classmethod from_proto(feature_view_proto: feast.core.FeatureView_pb2.FeatureView)[source]

Creates a feature view from a protobuf representation of a feature view

Parameters

feature_view_proto – A protobuf representation of a feature view

Returns

Returns a FeatureViewProto object based on the feature view protobuf

input: Union[feast.data_source.BigQuerySource, feast.data_source.FileSource]
is_valid()[source]

Validates the state of a feature view locally. Raises an exception if feature view is invalid.

last_updated_timestamp: Optional[google.protobuf.timestamp_pb2.Timestamp] = None
materialization_intervals: List[Tuple[datetime.datetime, datetime.datetime]]
property most_recent_end_time
name: str
online: bool
tags: Optional[Dict[str, str]]
to_proto()feast.core.FeatureView_pb2.FeatureView[source]

Converts an feature view object to its protobuf representation.

Returns

FeatureViewProto protobuf

ttl: Optional[datetime.timedelta]

feast.names module

feast.offline_store module

feast.online_response module

class feast.online_response.OnlineResponse(online_response_proto: feast.serving.ServingService_pb2.GetOnlineFeaturesResponse)[source]

Bases: object

Defines a online response in feast.

property field_values

Getter for GetOnlineResponse’s field_values.

to_dict()Dict[str, Any][source]

Converts GetOnlineFeaturesResponse features into a dictionary form.

feast.registry module

class feast.registry.GCSRegistryStore(uri: str)[source]

Bases: feast.registry.RegistryStore

get_registry_proto()[source]

Retrieves the registry proto from the registry path. If there is no file at that path, returns an empty registry proto.

Returns

Returns either the registry proto stored at the registry path, or an empty registry proto.

update_registry_proto(updater: Optional[Callable[[feast.core.Registry_pb2.Registry], feast.core.Registry_pb2.Registry]] = None)[source]

Updates the registry using the function passed in. If the registry proto has not been created yet this method will create it. This method writes to the registry path.

Parameters

updater – function that takes in the current registry proto and outputs the desired registry proto

class feast.registry.LocalRegistryStore(repo_path: pathlib.Path, registry_path_string: str)[source]

Bases: feast.registry.RegistryStore

get_registry_proto()[source]

Retrieves the registry proto from the registry path. If there is no file at that path, returns an empty registry proto.

Returns

Returns either the registry proto stored at the registry path, or an empty registry proto.

update_registry_proto(updater: Optional[Callable[[feast.core.Registry_pb2.Registry], feast.core.Registry_pb2.Registry]] = None)[source]

Updates the registry using the function passed in. If the registry proto has not been created yet this method will create it. This method writes to the registry path.

Parameters

updater – function that takes in the current registry proto and outputs the desired registry proto

class feast.registry.Registry(registry_path: str, repo_path: pathlib.Path, cache_ttl: datetime.timedelta)[source]

Bases: object

Registry: A registry allows for the management and persistence of feature definitions and related metadata.

apply_entity(entity: feast.entity.Entity, project: str)[source]

Registers a single entity with Feast

Parameters
  • entity – Entity that will be registered

  • project – Feast project that this entity belongs to

apply_feature_table(feature_table: feast.feature_table.FeatureTable, project: str)[source]

Registers a single feature table with Feast

Parameters
  • feature_table – Feature table that will be registered

  • project – Feast project that this feature table belongs to

apply_feature_view(feature_view: feast.feature_view.FeatureView, project: str)[source]

Registers a single feature view with Feast

Parameters
  • feature_view – Feature view that will be registered

  • project – Feast project that this feature view belongs to

cache_being_updated: bool = False
cached_registry_proto: Optional[feast.core.Registry_pb2.Registry] = None
cached_registry_proto_created: Optional[datetime.datetime] = None
cached_registry_proto_ttl: datetime.timedelta
delete_feature_table(name: str, project: str)[source]

Deletes a feature table or raises an exception if not found.

Parameters
  • name – Name of feature table

  • project – Feast project that this feature table belongs to

delete_feature_view(name: str, project: str)[source]

Deletes a feature view or raises an exception if not found.

Parameters
  • name – Name of feature view

  • project – Feast project that this feature view belongs to

get_entity(name: str, project: str, allow_cache: bool = False)feast.entity.Entity[source]

Retrieves an entity.

Parameters
  • name – Name of entity

  • project – Feast project that this entity belongs to

Returns

Returns either the specified entity, or raises an exception if none is found

get_feature_table(name: str, project: str)feast.feature_table.FeatureTable[source]

Retrieves a feature table.

Parameters
  • name – Name of feature table

  • project – Feast project that this feature table belongs to

Returns

Returns either the specified feature table, or raises an exception if none is found

get_feature_view(name: str, project: str)feast.feature_view.FeatureView[source]

Retrieves a feature view.

Parameters
  • name – Name of feature view

  • project – Feast project that this feature view belongs to

Returns

Returns either the specified feature view, or raises an exception if none is found

list_entities(project: str, allow_cache: bool = False)List[feast.entity.Entity][source]

Retrieve a list of entities from the registry

Parameters
  • allow_cache – Whether to allow returning entities from a cached registry

  • project – Filter entities based on project name

Returns

List of entities

list_feature_tables(project: str)List[feast.feature_table.FeatureTable][source]

Retrieve a list of feature tables from the registry

Parameters

project – Filter feature tables based on project name

Returns

List of feature tables

list_feature_views(project: str, allow_cache: bool = False)List[feast.feature_view.FeatureView][source]

Retrieve a list of feature views from the registry

Parameters
  • allow_cache – Allow returning feature views from the cached registry

  • project – Filter feature tables based on project name

Returns

List of feature views

refresh()[source]

Refreshes the state of the registry cache by fetching the registry state from the remote registry store.

class feast.registry.RegistryStore[source]

Bases: abc.ABC

RegistryStore: abstract base class implemented by specific backends (local file system, GCS) containing lower level methods used by the Registry class that are backend-specific.

abstract get_registry_proto()[source]

Retrieves the registry proto from the registry path. If there is no file at that path, returns an empty registry proto.

Returns

Returns either the registry proto stored at the registry path, or an empty registry proto.

abstract update_registry_proto(updater: Optional[Callable[[feast.core.Registry_pb2.Registry], feast.core.Registry_pb2.Registry]] = None)[source]

Updates the registry using the function passed in. If the registry proto has not been created yet this method will create it. This method writes to the registry path.

Parameters

updater – function that takes in the current registry proto and outputs the desired registry proto

feast.repo_config module

class feast.repo_config.DatastoreOnlineStoreConfig(*, type: typing_extensions.Literal[datastore] = 'datastore', project_id: pydantic.types.StrictStr = None)[source]

Bases: feast.repo_config.FeastBaseModel

Online store config for GCP Datastore

project_id: Optional[pydantic.types.StrictStr]

(optional) GCP Project Id

type: typing_extensions.Literal[datastore]

Online store type selector

class feast.repo_config.FeastBaseModel[source]

Bases: pydantic.main.BaseModel

Feast Pydantic Configuration Class

class Config[source]

Bases: object

arbitrary_types_allowed = True
extra = 'forbid'
exception feast.repo_config.FeastConfigError(error_message, config_path)[source]

Bases: Exception

class feast.repo_config.RegistryConfig(*, path: pydantic.types.StrictStr, cache_ttl_seconds: pydantic.types.StrictInt = 600)[source]

Bases: feast.repo_config.FeastBaseModel

Metadata Store Configuration. Configuration that relates to reading from and writing to the Feast registry.

cache_ttl_seconds: pydantic.types.StrictInt

The cache TTL is the amount of time registry state will be cached in memory. If this TTL is exceeded then the registry will be refreshed when any feature store method asks for access to registry state. The TTL can be set to infinity by setting TTL to 0 seconds, which means the cache will only be loaded once and will never expire. Users can manually refresh the cache by calling feature_store.refresh_registry()

Type

int

path: pydantic.types.StrictStr

//foo/bar

Type

str

Type

Path to metadata store. Can be a local path, or remote object storage path, e.g. gcs

class feast.repo_config.RepoConfig(*, registry: Union[pydantic.types.StrictStr, feast.repo_config.RegistryConfig] = 'data/registry.db', project: pydantic.types.StrictStr, provider: pydantic.types.StrictStr, online_store: Union[feast.repo_config.DatastoreOnlineStoreConfig, feast.repo_config.SqliteOnlineStoreConfig] = SqliteOnlineStoreConfig(type='sqlite', path='data/online.db'))[source]

Bases: feast.repo_config.FeastBaseModel

Repo config. Typically loaded from feature_store.yaml

get_registry_config()[source]
online_store: Union[feast.repo_config.DatastoreOnlineStoreConfig, feast.repo_config.SqliteOnlineStoreConfig]

Online store configuration (optional depending on provider)

Type

OnlineStoreConfig

project: pydantic.types.StrictStr

Feast project id. This can be any alphanumeric string up to 16 characters. You can have multiple independent feature repositories deployed to the same cloud provider account, as long as they have different project ids.

Type

str

provider: pydantic.types.StrictStr

local or gcp

Type

str

registry: Union[pydantic.types.StrictStr, feast.repo_config.RegistryConfig]

//foo/bar

Type

str

Type

Path to metadata store. Can be a local path, or remote object storage path, e.g. gcs

class feast.repo_config.SqliteOnlineStoreConfig(*, type: typing_extensions.Literal[sqlite] = 'sqlite', path: pydantic.types.StrictStr = 'data/online.db')[source]

Bases: feast.repo_config.FeastBaseModel

Online store config for local (SQLite-based) store

path: pydantic.types.StrictStr

(optional) Path to sqlite db

type: typing_extensions.Literal[sqlite]

Online store type selector

feast.repo_config.load_repo_config(repo_path: pathlib.Path)feast.repo_config.RepoConfig[source]

feast.repo_operations module

class feast.repo_operations.ParsedRepo(feature_tables, feature_views, entities)[source]

Bases: tuple

property entities

Alias for field number 2

property feature_tables

Alias for field number 0

property feature_views

Alias for field number 1

feast.repo_operations.apply_total(repo_config: feast.repo_config.RepoConfig, repo_path: pathlib.Path)[source]
feast.repo_operations.cli_check_repo(repo_path: pathlib.Path)[source]
feast.repo_operations.generate_project_name()str[source]

Generates a unique project name

feast.repo_operations.init_repo(repo_name: str, template: str)[source]
feast.repo_operations.parse_repo(repo_root: pathlib.Path)feast.repo_operations.ParsedRepo[source]

Collect feature table definitions from feature repo

feast.repo_operations.py_path_to_module(path: pathlib.Path, repo_root: pathlib.Path)str[source]
feast.repo_operations.registry_dump(repo_config: feast.repo_config.RepoConfig, repo_path: pathlib.Path)[source]

For debugging only: output contents of the metadata registry

feast.repo_operations.replace_str_in_file(file_path, match_str, sub_str)[source]
feast.repo_operations.teardown(repo_config: feast.repo_config.RepoConfig, repo_path: pathlib.Path)[source]

feast.telemetry module

class feast.telemetry.Telemetry[source]

Bases: object

log(function_name: str)[source]
property telemetry_id

feast.type_map module

feast.type_map.feast_value_type_to_python_type(field_value_proto: feast.types.Value_pb2.Value)Any[source]

Converts field value Proto to Dict and returns each field’s Feast Value Type value in their respective Python value.

Parameters

field_value_proto – Field value Proto

Returns

Python native type representation/version of the given field_value_proto

feast.type_map.pa_column_to_proto_column(feast_value_type: feast.value_type.ValueType, column: pyarrow.lib.ChunkedArray)List[feast.types.Value_pb2.Value][source]
feast.type_map.pa_column_to_timestamp_proto_column(column: pyarrow.lib.ChunkedArray)List[google.protobuf.timestamp_pb2.Timestamp][source]
feast.type_map.pa_to_feast_value_attr(pa_type: object)[source]

Returns the equivalent Feast ValueType string for the given pa.lib type.

Parameters

pa_type (object) – PyArrow type.

Returns

Feast attribute name in Feast ValueType string-ed representation.

Return type

str

feast.type_map.pa_to_feast_value_type(value: pyarrow.lib.ChunkedArray)feast.value_type.ValueType[source]
feast.type_map.pa_to_value_type(pa_type: object)[source]

Returns the equivalent Feast ValueType for the given pa.lib type.

Parameters

pa_type (object) – PyArrow type.

Returns

Feast ValueType.

Return type

feast.types.Value_pb2.ValueType

feast.type_map.python_type_to_feast_value_type(name: str, value, recurse: bool = True)feast.value_type.ValueType[source]

Finds the equivalent Feast Value Type for a Python value. Both native and Pandas types are supported. This function will recursively look for nested types when arrays are detected. All types must be homogenous.

Parameters
  • name – Name of the value or field

  • value – Value that will be inspected

  • recurse – Whether to recursively look for nested types in arrays

Returns

Feast Value Type

feast.type_map.python_value_to_proto_value(value: Any)feast.types.Value_pb2.Value[source]

feast.value_type module

class feast.value_type.ValueType(value)[source]

Bases: enum.Enum

Feature value type. Used to define data types in Feature Tables.

BOOL = 7
BOOL_LIST = 17
BYTES = 1
BYTES_LIST = 11
DOUBLE = 5
DOUBLE_LIST = 15
FLOAT = 6
FLOAT_LIST = 16
INT32 = 3
INT32_LIST = 13
INT64 = 4
INT64_LIST = 14
STRING = 2
STRING_LIST = 12
UNKNOWN = 0
to_tfx_schema_feature_type()[source]

feast.wait module

feast.wait.wait_retry_backoff(retry_fn: Callable[], Tuple[Any, bool]], timeout_secs: int = 0, timeout_msg: Optional[str] = 'Timeout while waiting for retry_fn() to return True', max_interval_secs: int = 60)Any[source]

Repeatedly try calling given retry_fn until it returns a True boolean success flag. Waits with a exponential backoff between retries until timeout when it throws TimeoutError. :param retry_fn: Callable that returns a result and a boolean success flag. :param timeout_secs: timeout in seconds to give up retrying and throw TimeoutError,

or 0 to retry perpetually.

Parameters
  • timeout_msg – Message to use when throwing TimeoutError.

  • max_interval_secs – max wait in seconds to wait between retries.

Returns

Returned Result from retry_fn() if success flag is True.

Module contents

class feast.BigQuerySource(event_timestamp_column: str, table_ref: Optional[str] = None, created_timestamp_column: Optional[str] = '', field_mapping: Optional[Dict[str, str]] = None, date_partition_column: Optional[str] = '', query: Optional[str] = None)[source]

Bases: feast.data_source.DataSource

property bigquery_options

Returns the bigquery options of this data source

get_table_query_string()str[source]

Returns a string that can directly be used to reference this table in SQL

property query
property table_ref
to_proto()feast.core.DataSource_pb2.DataSource[source]

Converts an DataSourceProto object to its protobuf representation.

class feast.Client(options: Optional[Dict[str, str]] = None, **kwargs)[source]

Bases: object

Feast Client: Used for creating, managing, and retrieving features.

apply(objects: Union[List[Union[feast.entity.Entity, feast.feature_table.FeatureTable]], feast.entity.Entity, feast.feature_table.FeatureTable], project: Optional[str] = None)[source]

Idempotently registers entities and feature tables with Feast Core. Either a single entity or feature table or a list can be provided.

Parameters

objects – List of entities and/or feature tables that will be registered

Examples

>>> from feast import Client
>>> from feast.entity import Entity
>>> from feast.value_type import ValueType
>>>
>>> feast_client = Client(core_url="localhost:6565")
>>> entity = Entity(
>>>     name="driver_entity",
>>>     description="Driver entity for car rides",
>>>     value_type=ValueType.STRING,
>>>     labels={
>>>         "key": "val"
>>>     }
>>> )
>>> feast_client.apply(entity)
apply_entity(entities: Union[List[feast.entity.Entity], feast.entity.Entity], project: Optional[str] = None)[source]

Deprecated. Please see apply().

apply_feature_table(feature_tables: Union[List[feast.feature_table.FeatureTable], feast.feature_table.FeatureTable], project: Optional[str] = None)[source]

Deprecated. Please see apply().

archive_project(project)[source]

Archives a project. Project will still continue to function for ingestion and retrieval, but will be in a read-only state. It will also not be visible from the Core API for management purposes.

Parameters

project – Name of project to archive

property config
property core_secure

Retrieve Feast Core client-side SSL/TLS setting

Returns

Whether client-side SSL/TLS is enabled

property core_url

Retrieve Feast Core URL

Returns

Feast Core URL string

create_project(project: str)[source]

Creates a Feast project

Parameters

project – Name of project

delete_feature_table(name: str, project: Optional[str] = None)None[source]

Deletes a feature table.

Parameters
  • project – Feast project that this feature table belongs to

  • name – Name of feature table

get_entity(name: str, project: Optional[str] = None)feast.entity.Entity[source]

Retrieves an entity.

Parameters
  • project – Feast project that this entity belongs to

  • name – Name of entity

Returns

Returns either the specified entity, or raises an exception if none is found

get_feature_table(name: str, project: Optional[str] = None)feast.feature_table.FeatureTable[source]

Retrieves a feature table.

Parameters
  • project – Feast project that this feature table belongs to

  • name – Name of feature table

Returns

Returns either the specified feature table, or raises an exception if none is found

get_online_features(feature_refs: List[str], entity_rows: List[Dict[str, Any]], project: Optional[str] = None)feast.online_response.OnlineResponse[source]

Retrieves the latest online feature data from Feast Serving. :param feature_refs: List of feature references that will be returned for each entity.

Each feature reference should have the following format: “feature_table:feature” where “feature_table” & “feature” refer to the feature and feature table names respectively. Only the feature name is required.

Parameters
  • entity_rows – A list of dictionaries where each key-value is an entity-name, entity-value pair.

  • project – Optionally specify the the project override. If specified, uses given project for retrieval. Overrides the projects specified in Feature References if also are specified.

Returns

GetOnlineFeaturesResponse containing the feature data in records. Each EntityRow provided will yield one record, which contains data fields with data value and field status metadata (if included).

Examples

>>> from feast import Client
>>>
>>> feast_client = Client(core_url="localhost:6565", serving_url="localhost:6566")
>>> feature_refs = ["sales:daily_transactions"]
>>> entity_rows = [{"customer_id": 0},{"customer_id": 1}]
>>>
>>> online_response = feast_client.get_online_features(
>>>     feature_refs, entity_rows, project="my_project")
>>> online_response_dict = online_response.to_dict()
>>> print(online_response_dict)
{'sales:daily_transactions': [1.1,1.2], 'sales:customer_id': [0,1]}
ingest(feature_table: Union[str, feast.feature_table.FeatureTable], source: Union[pandas.core.frame.DataFrame, str], project: Optional[str] = None, chunk_size: int = 10000, max_workers: int = 1, timeout: int = 120)None[source]

Batch load feature data into a FeatureTable.

Parameters
  • feature_table (typing.Union[str, feast.feature_table.FeatureTable]) – FeatureTable object or the string name of the feature table

  • source (typing.Union[pd.DataFrame, str]) –

    Either a file path or Pandas Dataframe to ingest into Feast Files that are currently supported:

    • parquet

    • csv

    • json

  • project – Feast project to locate FeatureTable

  • chunk_size (int) – Amount of rows to load and ingest at a time.

  • max_workers (int) – Number of worker processes to use to encode values.

  • timeout (int) – Timeout in seconds to wait for completion.

Examples

>>> from feast import Client
>>>
>>> client = Client(core_url="localhost:6565")
>>> ft_df = pd.DataFrame(
>>>         {
>>>            "datetime": [pd.datetime.now()],
>>>            "driver": [1001],
>>>            "rating": [4.3],
>>>         }
>>>     )
>>> client.set_project("project1")
>>>
>>> driver_ft = client.get_feature_table("driver")
>>> client.ingest(driver_ft, ft_df)
property job_service_secure

Retrieve Feast Job Service client-side SSL/TLS setting

Returns

Whether client-side SSL/TLS is enabled

property job_service_url

Retrieve Feast Job Service URL

Returns

Feast Job Service URL string

list_entities(project: Optional[str] = None, labels: Dict[str, str] = {})List[feast.entity.Entity][source]

Retrieve a list of entities from Feast Core

Parameters
  • project – Filter entities based on project name

  • labels – User-defined labels that these entities are associated with

Returns

List of entities

list_feature_tables(project: Optional[str] = None, labels: Dict[str, str] = {})List[feast.feature_table.FeatureTable][source]

Retrieve a list of feature tables from Feast Core

Parameters

project – Filter feature tables based on project name

Returns

List of feature tables

list_features_by_ref(project: Optional[str] = None, entities: List[str] = [], labels: Dict[str, str] = {})Dict[feast.feature.FeatureRef, feast.feature.Feature][source]

Retrieve a dictionary of feature reference to feature from Feast Core based on filters provided.

Parameters
  • project – Feast project that these features belongs to

  • entities – Feast entity that these features are associated with

  • labels – Feast labels that these features are associated with

Returns

features>

Return type

Dictionary of <feature references

Examples

>>> from feast import Client
>>>
>>> feast_client = Client(core_url="localhost:6565")
>>> features = feast_client.list_features(project="test_project", entities=["driver_id"], labels={"key1":"val1","key2":"val2"})
>>> print(features)
list_projects()List[str][source]

List all active Feast projects

Returns

List of project names

property project

Retrieve currently active project

Returns

Project name

property serving_secure

Retrieve Feast Serving client-side SSL/TLS setting

Returns

Whether client-side SSL/TLS is enabled

property serving_url

Retrieve Feast Serving URL

Returns

Feast Serving URL string

set_project(project: Optional[str] = None)[source]

Set currently active Feast project

Parameters

project – Project to set as active. If unset, will reset to the default project.

version(sdk_only=False)[source]

Returns version information from Feast Core and Feast Serving

class feast.Entity(name: str, value_type: feast.value_type.ValueType, description: str = '', join_key: Optional[str] = None, labels: Optional[MutableMapping[str, str]] = None)[source]

Bases: object

Represents a collection of entities and associated metadata.

property created_timestamp

Returns the created_timestamp of this entity

property description

Returns the description of this entity

classmethod from_dict(entity_dict)[source]

Creates an entity from a dict

Parameters

entity_dict – A dict representation of an entity

Returns

Returns a EntityV2 object based on the entity dict

classmethod from_proto(entity_proto: feast.core.Entity_pb2.Entity)[source]

Creates an entity from a protobuf representation of an entity

Parameters

entity_proto – A protobuf representation of an entity

Returns

Returns a EntityV2 object based on the entity protobuf

classmethod from_yaml(yml: str)[source]

Creates an entity from a YAML string body or a file path

Parameters

yml – Either a file path containing a yaml file or a YAML string

Returns

Returns a EntityV2 object based on the YAML file

is_valid()[source]

Validates the state of a entity locally. Raises an exception if entity is invalid.

property join_key

Returns the join key of this entity

property labels

Returns the labels of this entity. This is the user defined metadata defined as a dictionary.

property last_updated_timestamp

Returns the last_updated_timestamp of this entity

property name

Returns the name of this entity

to_dict()Dict[source]

Converts entity to dict

Returns

Dictionary object representation of entity

to_proto()feast.core.Entity_pb2.Entity[source]

Converts an entity object to its protobuf representation

Returns

EntityV2Proto protobuf

to_spec_proto()feast.core.Entity_pb2.EntitySpecV2[source]

Converts an EntityV2 object to its protobuf representation. Used when passing EntitySpecV2 object to Feast request.

Returns

EntitySpecV2 protobuf

to_yaml()[source]

Converts a entity to a YAML string.

Returns

Entity string returned in YAML format

property value_type

Returns the type of this entity

class feast.Feature(name: str, dtype: feast.value_type.ValueType, labels: Optional[MutableMapping[str, str]] = None)[source]

Bases: object

Feature field type

property dtype

Getter for data type of this field

classmethod from_proto(feature_proto: feast.core.Feature_pb2.FeatureSpecV2)[source]
Parameters

feature_proto – FeatureSpecV2 protobuf object

Returns

Feature object

property labels

Getter for labels of this field

property name

Getter for name of this field

to_proto()feast.core.Feature_pb2.FeatureSpecV2[source]

Converts Feature object to its Protocol Buffer representation

class feast.FeatureStore(repo_path: Optional[str] = None, config: Optional[feast.repo_config.RepoConfig] = None)[source]

Bases: object

A FeatureStore object is used to define, create, and retrieve features.

apply(objects: Union[feast.entity.Entity, feast.feature_view.FeatureView, List[Union[feast.feature_view.FeatureView, feast.entity.Entity]]])[source]

Register objects to metadata store and update related infrastructure.

The apply method registers one or more definitions (e.g., Entity, FeatureView) and registers or updates these objects in the Feast registry. Once the registry has been updated, the apply method will update related infrastructure (e.g., create tables in an online store) in order to reflect these new definitions. All operations are idempotent, meaning they can safely be rerun.

Args: objects (List[Union[FeatureView, Entity]]): A list of FeatureView or Entity objects that should be

registered

Examples

Register a single Entity and FeatureView. >>> from feast.feature_store import FeatureStore >>> from feast import Entity, FeatureView, Feature, ValueType, FileSource >>> from datetime import timedelta >>> >>> fs = FeatureStore() >>> customer_entity = Entity(name=”customer”, value_type=ValueType.INT64, description=”customer entity”) >>> customer_feature_view = FeatureView( >>> name=”customer_fv”, >>> entities=[“customer”], >>> features=[Feature(name=”age”, dtype=ValueType.INT64)], >>> input=FileSource(path=”file.parquet”, event_timestamp_column=”timestamp”), >>> ttl=timedelta(days=1) >>> ) >>> fs.apply([customer_entity, customer_feature_view])

config: feast.repo_config.RepoConfig
delete_feature_view(name: str)[source]

Deletes a feature view or raises an exception if not found.

Parameters

name – Name of feature view

get_entity(name: str)feast.entity.Entity[source]

Retrieves an entity.

Parameters

name – Name of entity

Returns

Returns either the specified entity, or raises an exception if none is found

get_feature_view(name: str)feast.feature_view.FeatureView[source]

Retrieves a feature view.

Parameters

name – Name of feature view

Returns

Returns either the specified feature view, or raises an exception if none is found

get_historical_features(entity_df: Union[pandas.core.frame.DataFrame, str], feature_refs: List[str])feast.infra.offline_stores.offline_store.RetrievalJob[source]

Enrich an entity dataframe with historical feature values for either training or batch scoring.

This method joins historical feature data from one or more feature views to an entity dataframe by using a time travel join.

Each feature view is joined to the entity dataframe using all entities configured for the respective feature view. All configured entities must be available in the entity dataframe. Therefore, the entity dataframe must contain all entities found in all feature views, but the individual feature views can have different entities.

Time travel is based on the configured TTL for each feature view. A shorter TTL will limit the amount of scanning that will be done in order to find feature data for a specific entity key. Setting a short TTL may result in null values being returned.

Parameters
  • entity_df (Union[pd.DataFrame, str]) – An entity dataframe is a collection of rows containing all entity columns (e.g., customer_id, driver_id) on which features need to be joined, as well as a event_timestamp column used to ensure point-in-time correctness. Either a Pandas DataFrame can be provided or a string SQL query. The query must be of a format supported by the configured offline store (e.g., BigQuery)

  • feature_refs – A list of features that should be retrieved from the offline store. Feature references are of the format “feature_view:feature”, e.g., “customer_fv:daily_transactions”.

Returns

RetrievalJob which can be used to materialize the results.

Examples

Retrieve historical features using a BigQuery SQL entity dataframe >>> from feast.feature_store import FeatureStore >>> >>> fs = FeatureStore(config=RepoConfig(provider=”gcp”)) >>> retrieval_job = fs.get_historical_features( >>> entity_df=”SELECT event_timestamp, order_id, customer_id from gcp_project.my_ds.customer_orders”, >>> feature_refs=[“customer:age”, “customer:avg_orders_1d”, “customer:avg_orders_7d”] >>> ) >>> feature_data = job.to_df() >>> model.fit(feature_data) # insert your modeling framework here.

get_online_features(feature_refs: List[str], entity_rows: List[Dict[str, Any]])feast.online_response.OnlineResponse[source]

Retrieves the latest online feature data.

Note: This method will download the full feature registry the first time it is run. If you are using a remote registry like GCS or S3 then that may take a few seconds. The registry remains cached up to a TTL duration (which can be set to infinitey). If the cached registry is stale (more time than the TTL has passed), then a new registry will be downloaded synchronously by this method. This download may introduce latency to online feature retrieval. In order to avoid synchronous downloads, please call refresh_registry() prior to the TTL being reached. Remember it is possible to set the cache TTL to infinity (cache forever).

Parameters
  • feature_refs – List of feature references that will be returned for each entity. Each feature reference should have the following format: “feature_table:feature” where “feature_table” & “feature” refer to the feature and feature table names respectively. Only the feature name is required.

  • entity_rows – A list of dictionaries where each key-value is an entity-name, entity-value pair.

Returns

OnlineResponse containing the feature data in records.

Examples

>>> from feast import FeatureStore
>>>
>>> store = FeatureStore(repo_path="...")
>>> feature_refs = ["sales:daily_transactions"]
>>> entity_rows = [{"customer_id": 0},{"customer_id": 1}]
>>>
>>> online_response = store.get_online_features(
>>>     feature_refs, entity_rows, project="my_project")
>>> online_response_dict = online_response.to_dict()
>>> print(online_response_dict)
{'sales:daily_transactions': [1.1,1.2], 'sales:customer_id': [0,1]}
list_entities(allow_cache: bool = False)List[feast.entity.Entity][source]

Retrieve a list of entities from the registry

Parameters

allow_cache (bool) – Whether to allow returning entities from a cached registry

Returns

List of entities

list_feature_views()List[feast.feature_view.FeatureView][source]

Retrieve a list of feature views from the registry

Returns

List of feature views

materialize(start_date: datetime.datetime, end_date: datetime.datetime, feature_views: Optional[List[str]] = None)None[source]

Materialize data from the offline store into the online store.

This method loads feature data in the specified interval from either the specified feature views, or all feature views if none are specified, into the online store where it is available for online serving.

Parameters
  • start_date (datetime) – Start date for time range of data to materialize into the online store

  • end_date (datetime) – End date for time range of data to materialize into the online store

  • feature_views (List[str]) – Optional list of feature view names. If selected, will only run materialization for the specified feature views.

Examples

Materialize all features into the online store over the interval from 3 hours ago to 10 minutes ago. >>> from datetime import datetime, timedelta >>> from feast.feature_store import FeatureStore >>> >>> fs = FeatureStore(config=RepoConfig(provider=”gcp”)) >>> fs.materialize( >>> start_date=datetime.utcnow() - timedelta(hours=3), end_date=datetime.utcnow() - timedelta(minutes=10) >>> )

materialize_incremental(end_date: datetime.datetime, feature_views: Optional[List[str]] = None)None[source]

Materialize incremental new data from the offline store into the online store.

This method loads incremental new feature data up to the specified end time from either the specified feature views, or all feature views if none are specified, into the online store where it is available for online serving. The start time of the interval materialized is either the most recent end time of a prior materialization or (now - ttl) if no such prior materialization exists.

Parameters
  • end_date (datetime) – End date for time range of data to materialize into the online store

  • feature_views (List[str]) – Optional list of feature view names. If selected, will only run materialization for the specified feature views.

Examples

Materialize all features into the online store up to 5 minutes ago. >>> from datetime import datetime, timedelta >>> from feast.feature_store import FeatureStore >>> >>> fs = FeatureStore(config=RepoConfig(provider=”gcp”, registry=”gs://my-fs/”, project=”my_fs_proj”)) >>> fs.materialize_incremental(end_date=datetime.utcnow() - timedelta(minutes=5))

property project
refresh_registry()[source]

Fetches and caches a copy of the feature registry in memory.

Explicitly calling this method allows for direct control of the state of the registry cache. Every time this method is called the complete registry state will be retrieved from the remote registry store backend (e.g., GCS, S3), and the cache timer will be reset. If refresh_registry() is run before get_online_features() is called, then get_online_feature() will use the cached registry instead of retrieving (and caching) the registry itself.

Additionally, the TTL for the registry cache can be set to infinity (by setting it to 0), which means that refresh_registry() will become the only way to update the cached registry. If the TTL is set to a value greater than 0, then once the cache becomes stale (more time than the TTL has passed), a new cache will be downloaded synchronously, which may increase latencies if the triggering method is get_online_features()

repo_path: pathlib.Path
version()str[source]

Returns the version of the current Feast SDK/CLI

class feast.FeatureTable(name: str, entities: List[str], features: List[feast.feature.Feature], batch_source: Optional[Union[feast.data_source.BigQuerySource, feast.data_source.FileSource]] = None, stream_source: Optional[Union[feast.data_source.KafkaSource, feast.data_source.KinesisSource]] = None, max_age: Optional[google.protobuf.duration_pb2.Duration] = None, labels: Optional[MutableMapping[str, str]] = None)[source]

Bases: object

Represents a collection of features and associated metadata.

add_feature(feature: feast.feature.Feature)[source]

Adds a new feature to the feature table.

property batch_source

Returns the batch source of this feature table

property created_timestamp

Returns the created_timestamp of this feature table

property entities

Returns the entities of this feature table

property features

Returns the features of this feature table

classmethod from_dict(ft_dict)[source]

Creates a feature table from a dict

Parameters

ft_dict – A dict representation of a feature table

Returns

Returns a FeatureTable object based on the feature table dict

classmethod from_proto(feature_table_proto: feast.core.FeatureTable_pb2.FeatureTable)[source]

Creates a feature table from a protobuf representation of a feature table

Parameters

feature_table_proto – A protobuf representation of a feature table

Returns

Returns a FeatureTableProto object based on the feature table protobuf

classmethod from_yaml(yml: str)[source]

Creates a feature table from a YAML string body or a file path

Parameters

yml – Either a file path containing a yaml file or a YAML string

Returns

Returns a FeatureTable object based on the YAML file

is_valid()[source]

Validates the state of a feature table locally. Raises an exception if feature table is invalid.

property labels

Returns the labels of this feature table. This is the user defined metadata defined as a dictionary.

property last_updated_timestamp

Returns the last_updated_timestamp of this feature table

property max_age

Returns the maximum age of this feature table. This is the total maximum amount of staleness that will be allowed during feature retrieval for each specific feature that is looked up.

property name

Returns the name of this feature table

property stream_source

Returns the stream source of this feature table

to_dict()Dict[source]

Converts feature table to dict

Returns

Dictionary object representation of feature table

to_proto()feast.core.FeatureTable_pb2.FeatureTable[source]

Converts an feature table object to its protobuf representation

Returns

FeatureTableProto protobuf

to_spec_proto()feast.core.FeatureTable_pb2.FeatureTableSpec[source]

Converts an FeatureTableProto object to its protobuf representation. Used when passing FeatureTableSpecProto object to Feast request.

Returns

FeatureTableSpecProto protobuf

to_yaml()[source]

Converts a feature table to a YAML string.

Returns

Feature table string returned in YAML format

class feast.FeatureView(name: str, entities: List[str], features: List[feast.feature.Feature], ttl: Optional[Union[google.protobuf.duration_pb2.Duration, datetime.timedelta]], input: Union[feast.data_source.BigQuerySource, feast.data_source.FileSource], tags: Optional[Dict[str, str]] = None, online: bool = True)[source]

Bases: object

A FeatureView defines a logical grouping of serveable features.

created_timestamp: Optional[google.protobuf.timestamp_pb2.Timestamp] = None
entities: List[str]
features: List[feast.feature.Feature]
classmethod from_proto(feature_view_proto: feast.core.FeatureView_pb2.FeatureView)[source]

Creates a feature view from a protobuf representation of a feature view

Parameters

feature_view_proto – A protobuf representation of a feature view

Returns

Returns a FeatureViewProto object based on the feature view protobuf

input: Union[feast.data_source.BigQuerySource, feast.data_source.FileSource]
is_valid()[source]

Validates the state of a feature view locally. Raises an exception if feature view is invalid.

last_updated_timestamp: Optional[google.protobuf.timestamp_pb2.Timestamp] = None
materialization_intervals: List[Tuple[datetime.datetime, datetime.datetime]]
property most_recent_end_time
name: str
online: bool
tags: Optional[Dict[str, str]]
to_proto()feast.core.FeatureView_pb2.FeatureView[source]

Converts an feature view object to its protobuf representation.

Returns

FeatureViewProto protobuf

ttl: Optional[datetime.timedelta]
class feast.FileSource(event_timestamp_column: str, file_url: Optional[str] = None, path: Optional[str] = None, file_format: Optional[feast.data_format.FileFormat] = None, created_timestamp_column: Optional[str] = '', field_mapping: Optional[Dict[str, str]] = None, date_partition_column: Optional[str] = '')[source]

Bases: feast.data_source.DataSource

property file_options

Returns the file options of this data source

property path

Returns the file path of this feature data source

to_proto()feast.core.DataSource_pb2.DataSource[source]

Converts an DataSourceProto object to its protobuf representation.

class feast.KafkaSource(event_timestamp_column: str, bootstrap_servers: str, message_format: feast.data_format.StreamFormat, topic: str, created_timestamp_column: Optional[str] = '', field_mapping: Optional[Dict[str, str]] = {}, date_partition_column: Optional[str] = '')[source]

Bases: feast.data_source.DataSource

property kafka_options

Returns the kafka options of this data source

to_proto()feast.core.DataSource_pb2.DataSource[source]

Converts an DataSourceProto object to its protobuf representation.

class feast.KinesisSource(event_timestamp_column: str, created_timestamp_column: str, record_format: feast.data_format.StreamFormat, region: str, stream_name: str, field_mapping: Optional[Dict[str, str]] = {}, date_partition_column: Optional[str] = '')[source]

Bases: feast.data_source.DataSource

property kinesis_options

Returns the kinesis options of this data source

to_proto()feast.core.DataSource_pb2.DataSource[source]

Converts an DataSourceProto object to its protobuf representation.

class feast.RepoConfig(*, registry: Union[pydantic.types.StrictStr, feast.repo_config.RegistryConfig] = 'data/registry.db', project: pydantic.types.StrictStr, provider: pydantic.types.StrictStr, online_store: Union[feast.repo_config.DatastoreOnlineStoreConfig, feast.repo_config.SqliteOnlineStoreConfig] = SqliteOnlineStoreConfig(type='sqlite', path='data/online.db'))[source]

Bases: feast.repo_config.FeastBaseModel

Repo config. Typically loaded from feature_store.yaml

get_registry_config()[source]
online_store: Union[feast.repo_config.DatastoreOnlineStoreConfig, feast.repo_config.SqliteOnlineStoreConfig]

Online store configuration (optional depending on provider)

Type

OnlineStoreConfig

project: pydantic.types.StrictStr

Feast project id. This can be any alphanumeric string up to 16 characters. You can have multiple independent feature repositories deployed to the same cloud provider account, as long as they have different project ids.

Type

str

provider: pydantic.types.StrictStr

local or gcp

Type

str

registry: Union[pydantic.types.StrictStr, feast.repo_config.RegistryConfig]

//foo/bar

Type

str

Type

Path to metadata store. Can be a local path, or remote object storage path, e.g. gcs

class feast.SourceType(value)[source]

Bases: enum.Enum

DataSource value type. Used to define source types in DataSource.

BATCH_BIGQUERY = 2
BATCH_FILE = 1
STREAM_KAFKA = 3
STREAM_KINESIS = 4
UNKNOWN = 0
class feast.ValueType(value)[source]

Bases: enum.Enum

Feature value type. Used to define data types in Feature Tables.

BOOL = 7
BOOL_LIST = 17
BYTES = 1
BYTES_LIST = 11
DOUBLE = 5
DOUBLE_LIST = 15
FLOAT = 6
FLOAT_LIST = 16
INT32 = 3
INT32_LIST = 13
INT64 = 4
INT64_LIST = 14
STRING = 2
STRING_LIST = 12
UNKNOWN = 0
to_tfx_schema_feature_type()[source]