feast.infra.online_stores package

Submodules

feast.infra.online_stores.datastore module

class feast.infra.online_stores.datastore.DatastoreOnlineStore[source]

Bases: feast.infra.online_stores.online_store.OnlineStore

OnlineStore is an object used for all interaction between Feast and the service used for offline storage of features.

online_read(config: feast.repo_config.RepoConfig, table: feast.feature_view.FeatureView, entity_keys: List[feast.types.EntityKey_pb2.EntityKey], requested_features: Optional[List[str]] = None) List[Tuple[Optional[datetime.datetime], Optional[Dict[str, feast.types.Value_pb2.Value]]]][source]

Read feature values given an Entity Key. This is a low level interface, not expected to be used by the users directly.

Parameters
  • config – The RepoConfig for the current FeatureStore.

  • table – Feast FeatureView

  • entity_keys – a list of entity keys that should be read from the FeatureStore.

  • requested_features – (Optional) A subset of the features that should be read from the FeatureStore.

Returns

Data is returned as a list, one item per entity key in the original order as the entity_keys argument. Each item in the list is a tuple of event_ts for the row, and the feature data as a dict from feature names to values. Values are returned as Value proto message.

online_write_batch(config: feast.repo_config.RepoConfig, table: feast.feature_view.FeatureView, data: List[Tuple[feast.types.EntityKey_pb2.EntityKey, Dict[str, feast.types.Value_pb2.Value], datetime.datetime, Optional[datetime.datetime]]], progress: Optional[Callable[[int], Any]]) None[source]

Write a batch of feature rows to the online store. This is a low level interface, not expected to be used by the users directly.

If a tz-naive timestamp is passed to this method, it should be assumed to be UTC by implementors.

Parameters
  • config – The RepoConfig for the current FeatureStore.

  • table – Feast FeatureView

  • data – a list of quadruplets containing Feature data. Each quadruplet contains an Entity Key,

  • values (a dict containing feature) –

  • row (an event timestamp for the) –

  • and

  • exists. (the created timestamp for the row if it) –

  • progress – Optional function to be called once every mini-batch of rows is written to

  • progress. (the online store. Can be used to display) –

teardown(config: feast.repo_config.RepoConfig, tables: Sequence[feast.feature_view.FeatureView], entities: Sequence[feast.entity.Entity])[source]
update(config: feast.repo_config.RepoConfig, tables_to_delete: Sequence[feast.feature_view.FeatureView], tables_to_keep: Sequence[feast.feature_view.FeatureView], entities_to_delete: Sequence[feast.entity.Entity], entities_to_keep: Sequence[feast.entity.Entity], partial: bool)[source]
class feast.infra.online_stores.datastore.DatastoreOnlineStoreConfig(*, type: typing_extensions.Literal[datastore] = 'datastore', project_id: pydantic.types.StrictStr = None, namespace: pydantic.types.StrictStr = None, write_concurrency: pydantic.types.PositiveInt = 40, write_batch_size: pydantic.types.PositiveInt = 50)[source]

Bases: feast.repo_config.FeastConfigBaseModel

Online store config for GCP Datastore

namespace: Optional[pydantic.types.StrictStr]

(optional) Datastore namespace

project_id: Optional[pydantic.types.StrictStr]

(optional) GCP Project Id

type: typing_extensions.Literal[datastore]

Online store type selector

write_batch_size: Optional[pydantic.types.PositiveInt]

(optional) Amount of feature rows per batch being written into Datastore

write_concurrency: Optional[pydantic.types.PositiveInt]

(optional) Amount of threads to use when writing batches of feature rows into Datastore

class feast.infra.online_stores.datastore.DatastoreTable(project: str, name: str, project_id: Optional[str] = None, namespace: Optional[str] = None)[source]

Bases: feast.infra.infra_object.InfraObject

A Datastore table managed by Feast.

project

The Feast project of the table.

Type

str

name

The name of the table.

project_id

The GCP project id.

Type

optional

namespace

Datastore namespace.

Type

optional

static from_infra_object_proto(infra_object_proto: feast.core.InfraObject_pb2.InfraObject) Any[source]

Returns an InfraObject created from a protobuf representation.

Parameters

infra_object_proto – A protobuf representation of an InfraObject.

Raises

FeastInvalidInfraObjectType – The type of InfraObject could not be identified.

static from_proto(datastore_table_proto: feast.core.DatastoreTable_pb2.DatastoreTable) Any[source]

Converts a protobuf representation of a subclass to an object of that subclass.

Parameters

infra_object_proto – A protobuf representation of an InfraObject.

Raises

FeastInvalidInfraObjectType – The type of InfraObject could not be identified.

namespace: Optional[str]
project: str
project_id: Optional[str]
teardown()[source]

Tears down the infrastructure object.

to_infra_object_proto() feast.core.InfraObject_pb2.InfraObject[source]

Converts an InfraObject to its protobuf representation, wrapped in an InfraObjectProto.

to_proto() Any[source]

Converts an InfraObject to its protobuf representation.

update()[source]

Deploys or updates the infrastructure object.

feast.infra.online_stores.dynamodb module

class feast.infra.online_stores.dynamodb.DynamoDBOnlineStore[source]

Bases: feast.infra.online_stores.online_store.OnlineStore

Online feature store for AWS DynamoDB.

_dynamodb_client

Boto3 DynamoDB client.

_dynamodb_resource

Boto3 DynamoDB resource.

online_read(config: feast.repo_config.RepoConfig, table: feast.feature_view.FeatureView, entity_keys: List[feast.types.EntityKey_pb2.EntityKey], requested_features: Optional[List[str]] = None) List[Tuple[Optional[datetime.datetime], Optional[Dict[str, feast.types.Value_pb2.Value]]]][source]

Retrieve feature values from the online DynamoDB store.

Parameters
  • config – The RepoConfig for the current FeatureStore.

  • table – Feast FeatureView.

  • entity_keys – a list of entity keys that should be read from the FeatureStore.

online_write_batch(config: feast.repo_config.RepoConfig, table: feast.feature_view.FeatureView, data: List[Tuple[feast.types.EntityKey_pb2.EntityKey, Dict[str, feast.types.Value_pb2.Value], datetime.datetime, Optional[datetime.datetime]]], progress: Optional[Callable[[int], Any]]) None[source]

Write a batch of feature rows to online DynamoDB store.

Note: This method applies a batch_writer to automatically handle any unprocessed items and resend them as needed, this is useful if you’re loading a lot of data at a time.

Parameters
  • config – The RepoConfig for the current FeatureStore.

  • table – Feast FeatureView.

  • data – a list of quadruplets containing Feature data. Each quadruplet contains an Entity Key,

  • values (a dict containing feature) –

  • row (an event timestamp for the) –

  • and

  • exists. (the created timestamp for the row if it) –

  • progress – Optional function to be called once every mini-batch of rows is written to

  • progress. (the online store. Can be used to display) –

teardown(config: feast.repo_config.RepoConfig, tables: Sequence[feast.feature_view.FeatureView], entities: Sequence[feast.entity.Entity])[source]

Delete tables from the DynamoDB Online Store.

Parameters
  • config – The RepoConfig for the current FeatureStore.

  • tables – Tables to delete from the feature repo.

update(config: feast.repo_config.RepoConfig, tables_to_delete: Sequence[feast.feature_view.FeatureView], tables_to_keep: Sequence[feast.feature_view.FeatureView], entities_to_delete: Sequence[feast.entity.Entity], entities_to_keep: Sequence[feast.entity.Entity], partial: bool)[source]

Update tables from the DynamoDB Online Store.

Parameters
  • config – The RepoConfig for the current FeatureStore.

  • tables_to_delete – Tables to delete from the DynamoDB Online Store.

  • tables_to_keep – Tables to keep in the DynamoDB Online Store.

class feast.infra.online_stores.dynamodb.DynamoDBOnlineStoreConfig(*, type: typing_extensions.Literal[dynamodb] = 'dynamodb', batch_size: int = 40, endpoint_url: str = None, region: pydantic.types.StrictStr, table_name_template: pydantic.types.StrictStr = '{project}.{table_name}')[source]

Bases: feast.repo_config.FeastConfigBaseModel

Online store config for DynamoDB store

batch_size: int

Number of items to retrieve in a DynamoDB BatchGetItem call.

endpoint_url: Optional[str]

8000

Type

DynamoDB local development endpoint Url, i.e. http

Type

//localhost

region: pydantic.types.StrictStr

AWS Region Name

table_name_template: pydantic.types.StrictStr

DynamoDB table name template

type: typing_extensions.Literal[dynamodb]

Online store type selector

class feast.infra.online_stores.dynamodb.DynamoDBTable(name: str, region: str, endpoint_url: Optional[str] = None)[source]

Bases: feast.infra.infra_object.InfraObject

A DynamoDB table managed by Feast.

name

The name of the table.

region

The region of the table.

Type

str

endpoint_url

Local DynamoDB Endpoint Url.

_dynamodb_client

Boto3 DynamoDB client.

_dynamodb_resource

Boto3 DynamoDB resource.

endpoint_url = None
static from_infra_object_proto(infra_object_proto: feast.core.InfraObject_pb2.InfraObject) Any[source]

Returns an InfraObject created from a protobuf representation.

Parameters

infra_object_proto – A protobuf representation of an InfraObject.

Raises

FeastInvalidInfraObjectType – The type of InfraObject could not be identified.

static from_proto(dynamodb_table_proto: feast.core.DynamoDBTable_pb2.DynamoDBTable) Any[source]

Converts a protobuf representation of a subclass to an object of that subclass.

Parameters

infra_object_proto – A protobuf representation of an InfraObject.

Raises

FeastInvalidInfraObjectType – The type of InfraObject could not be identified.

region: str
teardown()[source]

Tears down the infrastructure object.

to_infra_object_proto() feast.core.InfraObject_pb2.InfraObject[source]

Converts an InfraObject to its protobuf representation, wrapped in an InfraObjectProto.

to_proto() Any[source]

Converts an InfraObject to its protobuf representation.

update()[source]

Deploys or updates the infrastructure object.

feast.infra.online_stores.helpers module

feast.infra.online_stores.helpers.compute_entity_id(entity_key: feast.types.EntityKey_pb2.EntityKey) str[source]

Compute Entity id given Feast Entity Key for online stores. Remember that Entity here refers to EntityKeyProto which is used in some online stores to encode the keys. It has nothing to do with the Entity concept we have in Feast.

feast.infra.online_stores.helpers.get_online_store_from_config(online_store_config: Any) feast.infra.online_stores.online_store.OnlineStore[source]

Creates an online store corresponding to the given online store config.

feast.infra.online_stores.online_store module

class feast.infra.online_stores.online_store.OnlineStore[source]

Bases: abc.ABC

OnlineStore is an object used for all interaction between Feast and the service used for online storage of features.

abstract online_read(config: feast.repo_config.RepoConfig, table: feast.feature_view.FeatureView, entity_keys: List[feast.types.EntityKey_pb2.EntityKey], requested_features: Optional[List[str]] = None) List[Tuple[Optional[datetime.datetime], Optional[Dict[str, feast.types.Value_pb2.Value]]]][source]

Read feature values given an Entity Key. This is a low level interface, not expected to be used by the users directly.

Parameters
  • config – The RepoConfig for the current FeatureStore.

  • table – Feast FeatureView

  • entity_keys – a list of entity keys that should be read from the FeatureStore.

  • requested_features – (Optional) A subset of the features that should be read from the FeatureStore.

Returns

Data is returned as a list, one item per entity key in the original order as the entity_keys argument. Each item in the list is a tuple of event_ts for the row, and the feature data as a dict from feature names to values. Values are returned as Value proto message.

abstract online_write_batch(config: feast.repo_config.RepoConfig, table: feast.feature_view.FeatureView, data: List[Tuple[feast.types.EntityKey_pb2.EntityKey, Dict[str, feast.types.Value_pb2.Value], datetime.datetime, Optional[datetime.datetime]]], progress: Optional[Callable[[int], Any]]) None[source]

Write a batch of feature rows to the online store. This is a low level interface, not expected to be used by the users directly.

If a tz-naive timestamp is passed to this method, it should be assumed to be UTC by implementors.

Parameters
  • config – The RepoConfig for the current FeatureStore.

  • table – Feast FeatureView

  • data – a list of quadruplets containing Feature data. Each quadruplet contains an Entity Key,

  • values (a dict containing feature) –

  • row (an event timestamp for the) –

  • and

  • exists. (the created timestamp for the row if it) –

  • progress – Optional function to be called once every mini-batch of rows is written to

  • progress. (the online store. Can be used to display) –

plan(config: feast.repo_config.RepoConfig, desired_registry_proto: feast.core.Registry_pb2.Registry) List[feast.infra.infra_object.InfraObject][source]

Returns the set of InfraObjects required to support the desired registry.

Parameters
  • config – The RepoConfig for the current FeatureStore.

  • desired_registry_proto – The desired registry, in proto form.

abstract teardown(config: feast.repo_config.RepoConfig, tables: Sequence[feast.feature_view.FeatureView], entities: Sequence[feast.entity.Entity])[source]
abstract update(config: feast.repo_config.RepoConfig, tables_to_delete: Sequence[feast.feature_view.FeatureView], tables_to_keep: Sequence[feast.feature_view.FeatureView], entities_to_delete: Sequence[feast.entity.Entity], entities_to_keep: Sequence[feast.entity.Entity], partial: bool)[source]

feast.infra.online_stores.redis module

class feast.infra.online_stores.redis.RedisOnlineStore[source]

Bases: feast.infra.online_stores.online_store.OnlineStore

delete_entity_values(config: feast.repo_config.RepoConfig, join_keys: List[str])[source]
online_read(config: feast.repo_config.RepoConfig, table: feast.feature_view.FeatureView, entity_keys: List[feast.types.EntityKey_pb2.EntityKey], requested_features: Optional[List[str]] = None) List[Tuple[Optional[datetime.datetime], Optional[Dict[str, feast.types.Value_pb2.Value]]]][source]

Read feature values given an Entity Key. This is a low level interface, not expected to be used by the users directly.

Parameters
  • config – The RepoConfig for the current FeatureStore.

  • table – Feast FeatureView

  • entity_keys – a list of entity keys that should be read from the FeatureStore.

  • requested_features – (Optional) A subset of the features that should be read from the FeatureStore.

Returns

Data is returned as a list, one item per entity key in the original order as the entity_keys argument. Each item in the list is a tuple of event_ts for the row, and the feature data as a dict from feature names to values. Values are returned as Value proto message.

online_write_batch(config: feast.repo_config.RepoConfig, table: feast.feature_view.FeatureView, data: List[Tuple[feast.types.EntityKey_pb2.EntityKey, Dict[str, feast.types.Value_pb2.Value], datetime.datetime, Optional[datetime.datetime]]], progress: Optional[Callable[[int], Any]]) None[source]

Write a batch of feature rows to the online store. This is a low level interface, not expected to be used by the users directly.

If a tz-naive timestamp is passed to this method, it should be assumed to be UTC by implementors.

Parameters
  • config – The RepoConfig for the current FeatureStore.

  • table – Feast FeatureView

  • data – a list of quadruplets containing Feature data. Each quadruplet contains an Entity Key,

  • values (a dict containing feature) –

  • row (an event timestamp for the) –

  • and

  • exists. (the created timestamp for the row if it) –

  • progress – Optional function to be called once every mini-batch of rows is written to

  • progress. (the online store. Can be used to display) –

teardown(config: feast.repo_config.RepoConfig, tables: Sequence[feast.feature_view.FeatureView], entities: Sequence[feast.entity.Entity])[source]

We delete the keys in redis for tables/views being removed.

update(config: feast.repo_config.RepoConfig, tables_to_delete: Sequence[feast.feature_view.FeatureView], tables_to_keep: Sequence[feast.feature_view.FeatureView], entities_to_delete: Sequence[feast.entity.Entity], entities_to_keep: Sequence[feast.entity.Entity], partial: bool)[source]

Look for join_keys (list of entities) that are not in use anymore (usually this happens when the last feature view that was using specific compound key is deleted) and remove all features attached to this “join_keys”.

class feast.infra.online_stores.redis.RedisOnlineStoreConfig(*, type: typing_extensions.Literal[redis] = 'redis', redis_type: feast.infra.online_stores.redis.RedisType = RedisType.redis, connection_string: pydantic.types.StrictStr = 'localhost:6379', key_ttl_seconds: int = None)[source]

Bases: feast.repo_config.FeastConfigBaseModel

Online store config for Redis store

connection_string: pydantic.types.StrictStr

Connection string containing the host, port, and configuration parameters for Redis format: host:port,parameter1,parameter2 eg. redis:6379,db=0

key_ttl_seconds: Optional[int]

(Optional) redis key bin ttl (in seconds) for expiring entities

redis_type: feast.infra.online_stores.redis.RedisType

redis or redis_cluster

Type

Redis type

type: typing_extensions.Literal[redis]

Online store type selector

class feast.infra.online_stores.redis.RedisType(value)[source]

Bases: str, enum.Enum

An enumeration.

redis = 'redis'
redis_cluster = 'redis_cluster'

feast.infra.online_stores.sqlite module

class feast.infra.online_stores.sqlite.SqliteOnlineStore[source]

Bases: feast.infra.online_stores.online_store.OnlineStore

OnlineStore is an object used for all interaction between Feast and the service used for offline storage of features.

_conn

SQLite connection.

Type

Optional[sqlite3.Connection]

online_read(config: feast.repo_config.RepoConfig, table: feast.feature_view.FeatureView, entity_keys: List[feast.types.EntityKey_pb2.EntityKey], requested_features: Optional[List[str]] = None) List[Tuple[Optional[datetime.datetime], Optional[Dict[str, feast.types.Value_pb2.Value]]]][source]

Read feature values given an Entity Key. This is a low level interface, not expected to be used by the users directly.

Parameters
  • config – The RepoConfig for the current FeatureStore.

  • table – Feast FeatureView

  • entity_keys – a list of entity keys that should be read from the FeatureStore.

  • requested_features – (Optional) A subset of the features that should be read from the FeatureStore.

Returns

Data is returned as a list, one item per entity key in the original order as the entity_keys argument. Each item in the list is a tuple of event_ts for the row, and the feature data as a dict from feature names to values. Values are returned as Value proto message.

online_write_batch(config: feast.repo_config.RepoConfig, table: feast.feature_view.FeatureView, data: List[Tuple[feast.types.EntityKey_pb2.EntityKey, Dict[str, feast.types.Value_pb2.Value], datetime.datetime, Optional[datetime.datetime]]], progress: Optional[Callable[[int], Any]]) None[source]

Write a batch of feature rows to the online store. This is a low level interface, not expected to be used by the users directly.

If a tz-naive timestamp is passed to this method, it should be assumed to be UTC by implementors.

Parameters
  • config – The RepoConfig for the current FeatureStore.

  • table – Feast FeatureView

  • data – a list of quadruplets containing Feature data. Each quadruplet contains an Entity Key,

  • values (a dict containing feature) –

  • row (an event timestamp for the) –

  • and

  • exists. (the created timestamp for the row if it) –

  • progress – Optional function to be called once every mini-batch of rows is written to

  • progress. (the online store. Can be used to display) –

plan(config: feast.repo_config.RepoConfig, desired_registry_proto: feast.core.Registry_pb2.Registry) List[feast.infra.infra_object.InfraObject][source]

Returns the set of InfraObjects required to support the desired registry.

Parameters
  • config – The RepoConfig for the current FeatureStore.

  • desired_registry_proto – The desired registry, in proto form.

teardown(config: feast.repo_config.RepoConfig, tables: Sequence[feast.feature_view.FeatureView], entities: Sequence[feast.entity.Entity])[source]
update(config: feast.repo_config.RepoConfig, tables_to_delete: Sequence[feast.feature_view.FeatureView], tables_to_keep: Sequence[feast.feature_view.FeatureView], entities_to_delete: Sequence[feast.entity.Entity], entities_to_keep: Sequence[feast.entity.Entity], partial: bool)[source]
class feast.infra.online_stores.sqlite.SqliteOnlineStoreConfig(*, type: typing_extensions.Literal[sqlite, feast.infra.online_stores.sqlite.SqliteOnlineStore] = 'sqlite', path: pydantic.types.StrictStr = 'data/online.db')[source]

Bases: feast.repo_config.FeastConfigBaseModel

Online store config for local (SQLite-based) store

path: pydantic.types.StrictStr

(optional) Path to sqlite db

type: typing_extensions.Literal[sqlite, feast.infra.online_stores.sqlite.SqliteOnlineStore]

Online store type selector

class feast.infra.online_stores.sqlite.SqliteTable(path: str, name: str)[source]

Bases: feast.infra.infra_object.InfraObject

A Sqlite table managed by Feast.

path

The absolute path of the Sqlite file.

Type

str

name

The name of the table.

conn

SQLite connection.

Type

sqlite3.Connection

conn: sqlite3.Connection
static from_infra_object_proto(infra_object_proto: feast.core.InfraObject_pb2.InfraObject) Any[source]

Returns an InfraObject created from a protobuf representation.

Parameters

infra_object_proto – A protobuf representation of an InfraObject.

Raises

FeastInvalidInfraObjectType – The type of InfraObject could not be identified.

static from_proto(sqlite_table_proto: feast.core.SqliteTable_pb2.SqliteTable) Any[source]

Converts a protobuf representation of a subclass to an object of that subclass.

Parameters

infra_object_proto – A protobuf representation of an InfraObject.

Raises

FeastInvalidInfraObjectType – The type of InfraObject could not be identified.

path: str
teardown()[source]

Tears down the infrastructure object.

to_infra_object_proto() feast.core.InfraObject_pb2.InfraObject[source]

Converts an InfraObject to its protobuf representation, wrapped in an InfraObjectProto.

to_proto() Any[source]

Converts an InfraObject to its protobuf representation.

update()[source]

Deploys or updates the infrastructure object.

Module contents