feast.infra package

Submodules

feast.infra.gcp module

class feast.infra.gcp.GcpProvider(config: feast.repo_config.RepoConfig)[source]

Bases: feast.infra.provider.Provider

static get_historical_features(config: feast.repo_config.RepoConfig, feature_views: List[feast.feature_view.FeatureView], feature_refs: List[str], entity_df: Union[pandas.core.frame.DataFrame, str], registry: feast.registry.Registry, project: str) → feast.infra.offline_stores.offline_store.RetrievalJob[source]
materialize_single_feature_view(feature_view: feast.feature_view.FeatureView, start_date: datetime.datetime, end_date: datetime.datetime, registry: feast.registry.Registry, project: str) → None[source]
online_read(project: str, table: Union[feast.feature_table.FeatureTable, feast.feature_view.FeatureView], entity_keys: List[feast.types.EntityKey_pb2.EntityKey]) → List[Tuple[Optional[datetime.datetime], Optional[Dict[str, feast.types.Value_pb2.Value]]]][source]

Read feature values given an Entity Key. This is a low level interface, not expected to be used by the users directly.

Returns

Data is returned as a list, one item per entity key. Each item in the list is a tuple of event_ts for the row, and the feature data as a dict from feature names to values. Values are returned as Value proto message.

online_write_batch(project: str, table: Union[feast.feature_table.FeatureTable, feast.feature_view.FeatureView], data: List[Tuple[feast.types.EntityKey_pb2.EntityKey, Dict[str, feast.types.Value_pb2.Value], datetime.datetime, Optional[datetime.datetime]]], progress: Optional[Callable[[int], Any]]) → None[source]

Write a batch of feature rows to the online store. This is a low level interface, not expected to be used by the users directly.

If a tz-naive timestamp is passed to this method, it is assumed to be UTC.

Parameters

  • project – Feast project name

  • table – Feast FeatureTable

  • data – a list of quadruplets containing Feature data. Each quadruplet contains an Entity Key, a dict containing feature values, an event timestamp for the row, and the created timestamp for the row if it exists.

  • progress – Optional function to be called once every mini-batch of rows is written to the online store. Can be used to display progress.

teardown_infra(project: str, tables: Sequence[Union[feast.feature_table.FeatureTable, feast.feature_view.FeatureView]]) → None[source]

Tear down all cloud resources for a repo.

Parameters

  • project – Feast project to which tables belong

  • tables – Tables that are declared in the feature repo.

update_infra(project: str, tables_to_delete: Sequence[Union[feast.feature_table.FeatureTable, feast.feature_view.FeatureView]], tables_to_keep: Sequence[Union[feast.feature_table.FeatureTable, feast.feature_view.FeatureView]], partial: bool)[source]

Reconcile cloud resources with the objects declared in the feature repo.

Parameters

  • project – Project to which tables belong

  • tables_to_delete – Tables that were deleted from the feature repo, so provider needs to clean up the corresponding cloud resources.

  • tables_to_keep – Tables that are still in the feature repo. Depending on implementation, provider may or may not need to update the corresponding resources.

  • partial – if true, then tables_to_delete and tables_to_keep are not exhaustive lists. There may be other tables that are not touched by this update.

feast.infra.gcp.compute_datastore_entity_id(entity_key: feast.types.EntityKey_pb2.EntityKey) → str[source]

Compute Datastore Entity id given Feast Entity Key.

Remember that Datastore Entity is a concept from the Datastore data model, that has nothing to do with the Entity concept we have in Feast.

feast.infra.key_encoding_utils module

feast.infra.key_encoding_utils.serialize_entity_key(entity_key: feast.types.EntityKey_pb2.EntityKey) → bytes[source]

Serialize entity key to a bytestring so it can be used as a lookup key in a hash table.

We need this encoding to be stable; therefore we cannot just use protobuf serialization here since it does not guarantee that two proto messages containing the same data will serialize to the same byte string[1].

[1] https://developers.google.com/protocol-buffers/docs/encoding

feast.infra.local_sqlite module

feast.infra.provider module

class feast.infra.provider.Provider[source]

Bases: abc.ABC

abstract static get_historical_features(config: feast.repo_config.RepoConfig, feature_views: List[feast.feature_view.FeatureView], feature_refs: List[str], entity_df: Union[pandas.core.frame.DataFrame, str], registry: feast.registry.Registry, project: str) → feast.infra.offline_stores.offline_store.RetrievalJob[source]
abstract materialize_single_feature_view(feature_view: feast.feature_view.FeatureView, start_date: datetime.datetime, end_date: datetime.datetime, registry: feast.registry.Registry, project: str) → None[source]
abstract online_read(project: str, table: Union[feast.feature_table.FeatureTable, feast.feature_view.FeatureView], entity_keys: List[feast.types.EntityKey_pb2.EntityKey]) → List[Tuple[Optional[datetime.datetime], Optional[Dict[str, feast.types.Value_pb2.Value]]]][source]

Read feature values given an Entity Key. This is a low level interface, not expected to be used by the users directly.

Returns

Data is returned as a list, one item per entity key. Each item in the list is a tuple of event_ts for the row, and the feature data as a dict from feature names to values. Values are returned as Value proto message.

abstract online_write_batch(project: str, table: Union[feast.feature_table.FeatureTable, feast.feature_view.FeatureView], data: List[Tuple[feast.types.EntityKey_pb2.EntityKey, Dict[str, feast.types.Value_pb2.Value], datetime.datetime, Optional[datetime.datetime]]], progress: Optional[Callable[[int], Any]]) → None[source]

Write a batch of feature rows to the online store. This is a low level interface, not expected to be used by the users directly.

If a tz-naive timestamp is passed to this method, it is assumed to be UTC.

Parameters

  • project – Feast project name

  • table – Feast FeatureTable

  • data – a list of quadruplets containing Feature data. Each quadruplet contains an Entity Key, a dict containing feature values, an event timestamp for the row, and the created timestamp for the row if it exists.

  • progress – Optional function to be called once every mini-batch of rows is written to the online store. Can be used to display progress.

abstract teardown_infra(project: str, tables: Sequence[Union[feast.feature_table.FeatureTable, feast.feature_view.FeatureView]])[source]

Tear down all cloud resources for a repo.

Parameters

  • project – Feast project to which tables belong

  • tables – Tables that are declared in the feature repo.

abstract update_infra(project: str, tables_to_delete: Sequence[Union[feast.feature_table.FeatureTable, feast.feature_view.FeatureView]], tables_to_keep: Sequence[Union[feast.feature_table.FeatureTable, feast.feature_view.FeatureView]], partial: bool)[source]

Reconcile cloud resources with the objects declared in the feature repo.

Parameters

  • project – Project to which tables belong

  • tables_to_delete – Tables that were deleted from the feature repo, so provider needs to clean up the corresponding cloud resources.

  • tables_to_keep – Tables that are still in the feature repo. Depending on implementation, provider may or may not need to update the corresponding resources.

  • partial – if true, then tables_to_delete and tables_to_keep are not exhaustive lists. There may be other tables that are not touched by this update.

feast.infra.provider.get_provider(config: feast.repo_config.RepoConfig, repo_path: pathlib.Path) → feast.infra.provider.Provider[source]

Module contents