feast.infra.materialization package

Subpackages

Submodules

feast.infra.materialization.batch_materialization_engine module

class feast.infra.materialization.batch_materialization_engine.BatchMaterializationEngine(*, repo_config: feast.repo_config.RepoConfig, offline_store: feast.infra.offline_stores.offline_store.OfflineStore, online_store: feast.infra.online_stores.online_store.OnlineStore, **kwargs)[source]

Bases: abc.ABC

abstract materialize(registry: feast.registry.BaseRegistry, tasks: List[feast.infra.materialization.batch_materialization_engine.MaterializationTask]) List[feast.infra.materialization.batch_materialization_engine.MaterializationJob][source]

Materialize data from the offline store to the online store for this feature repo. :param registry: The feast registry containing the applied feature views. :param tasks: A list of individual materialization tasks.

Returns

A list of materialization jobs representing each task.

abstract teardown_infra(project: str, fvs: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities: Sequence[feast.entity.Entity])[source]

This method ensures that any infrastructure or resources set up by ``update()``are torn down.

abstract update(project: str, views_to_delete: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], views_to_keep: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities_to_delete: Sequence[feast.entity.Entity], entities_to_keep: Sequence[feast.entity.Entity])[source]

This method ensures that any necessary infrastructure or resources needed by the engine are set up ahead of materialization.

class feast.infra.materialization.batch_materialization_engine.MaterializationJob[source]

Bases: abc.ABC

MaterializationJob represents an ongoing or executed process that materializes data as per the definition of a materialization task.

abstract error() Optional[BaseException][source]
abstract job_id() str[source]
abstract should_be_retried() bool[source]
abstract status() feast.infra.materialization.batch_materialization_engine.MaterializationJobStatus[source]
task: feast.infra.materialization.batch_materialization_engine.MaterializationTask
abstract url() Optional[str][source]
class feast.infra.materialization.batch_materialization_engine.MaterializationJobStatus(value)[source]

Bases: enum.Enum

An enumeration.

AVAILABLE = 3
CANCELLED = 6
CANCELLING = 5
ERROR = 4
RUNNING = 2
SUCCEEDED = 7
WAITING = 1
class feast.infra.materialization.batch_materialization_engine.MaterializationTask(project: str, feature_view: Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView], start_time: datetime.datetime, end_time: datetime.datetime, tqdm_builder: Callable[[int], tqdm.std.tqdm])[source]

Bases: object

A MaterializationTask represents a unit of data that needs to be materialized from an offline store to an online store.

end_time: datetime.datetime
feature_view: Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]
project: str
start_time: datetime.datetime
tqdm_builder: Callable[[int], tqdm.std.tqdm]

feast.infra.materialization.local_engine module

class feast.infra.materialization.local_engine.LocalMaterializationEngine(*, repo_config: feast.repo_config.RepoConfig, offline_store: feast.infra.offline_stores.offline_store.OfflineStore, online_store: feast.infra.online_stores.online_store.OnlineStore, **kwargs)[source]

Bases: feast.infra.materialization.batch_materialization_engine.BatchMaterializationEngine

materialize(registry, tasks: List[feast.infra.materialization.batch_materialization_engine.MaterializationTask]) List[feast.infra.materialization.batch_materialization_engine.MaterializationJob][source]

Materialize data from the offline store to the online store for this feature repo. :param registry: The feast registry containing the applied feature views. :param tasks: A list of individual materialization tasks.

Returns

A list of materialization jobs representing each task.

teardown_infra(project: str, fvs: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities: Sequence[feast.entity.Entity])[source]

This method ensures that any infrastructure or resources set up by ``update()``are torn down.

update(project: str, views_to_delete: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], views_to_keep: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities_to_delete: Sequence[feast.entity.Entity], entities_to_keep: Sequence[feast.entity.Entity])[source]

This method ensures that any necessary infrastructure or resources needed by the engine are set up ahead of materialization.

class feast.infra.materialization.local_engine.LocalMaterializationEngineConfig(*, type: Literal['local'] = 'local')[source]

Bases: feast.repo_config.FeastConfigBaseModel

Batch Materialization Engine config for local in-process engine

type: Literal['local']

Type selector

class feast.infra.materialization.local_engine.LocalMaterializationJob(job_id: str, status: feast.infra.materialization.batch_materialization_engine.MaterializationJobStatus, error: Union[BaseException, NoneType] = None)[source]

Bases: feast.infra.materialization.batch_materialization_engine.MaterializationJob

error() Optional[BaseException][source]
job_id() str[source]
should_be_retried() bool[source]
status() feast.infra.materialization.batch_materialization_engine.MaterializationJobStatus[source]
task: feast.infra.materialization.batch_materialization_engine.MaterializationTask
url() Optional[str][source]

Module contents

class feast.infra.materialization.BatchMaterializationEngine(*, repo_config: feast.repo_config.RepoConfig, offline_store: feast.infra.offline_stores.offline_store.OfflineStore, online_store: feast.infra.online_stores.online_store.OnlineStore, **kwargs)[source]

Bases: abc.ABC

abstract materialize(registry: feast.registry.BaseRegistry, tasks: List[feast.infra.materialization.batch_materialization_engine.MaterializationTask]) List[feast.infra.materialization.batch_materialization_engine.MaterializationJob][source]

Materialize data from the offline store to the online store for this feature repo. :param registry: The feast registry containing the applied feature views. :param tasks: A list of individual materialization tasks.

Returns

A list of materialization jobs representing each task.

abstract teardown_infra(project: str, fvs: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities: Sequence[feast.entity.Entity])[source]

This method ensures that any infrastructure or resources set up by ``update()``are torn down.

abstract update(project: str, views_to_delete: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], views_to_keep: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities_to_delete: Sequence[feast.entity.Entity], entities_to_keep: Sequence[feast.entity.Entity])[source]

This method ensures that any necessary infrastructure or resources needed by the engine are set up ahead of materialization.

class feast.infra.materialization.LocalMaterializationEngine(*, repo_config: feast.repo_config.RepoConfig, offline_store: feast.infra.offline_stores.offline_store.OfflineStore, online_store: feast.infra.online_stores.online_store.OnlineStore, **kwargs)[source]

Bases: feast.infra.materialization.batch_materialization_engine.BatchMaterializationEngine

materialize(registry, tasks: List[feast.infra.materialization.batch_materialization_engine.MaterializationTask]) List[feast.infra.materialization.batch_materialization_engine.MaterializationJob][source]

Materialize data from the offline store to the online store for this feature repo. :param registry: The feast registry containing the applied feature views. :param tasks: A list of individual materialization tasks.

Returns

A list of materialization jobs representing each task.

teardown_infra(project: str, fvs: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities: Sequence[feast.entity.Entity])[source]

This method ensures that any infrastructure or resources set up by ``update()``are torn down.

update(project: str, views_to_delete: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], views_to_keep: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities_to_delete: Sequence[feast.entity.Entity], entities_to_keep: Sequence[feast.entity.Entity])[source]

This method ensures that any necessary infrastructure or resources needed by the engine are set up ahead of materialization.

class feast.infra.materialization.LocalMaterializationJob(job_id: str, status: feast.infra.materialization.batch_materialization_engine.MaterializationJobStatus, error: Union[BaseException, NoneType] = None)[source]

Bases: feast.infra.materialization.batch_materialization_engine.MaterializationJob

error() Optional[BaseException][source]
job_id() str[source]
should_be_retried() bool[source]
status() feast.infra.materialization.batch_materialization_engine.MaterializationJobStatus[source]
task: feast.infra.materialization.batch_materialization_engine.MaterializationTask
url() Optional[str][source]
class feast.infra.materialization.MaterializationJob[source]

Bases: abc.ABC

MaterializationJob represents an ongoing or executed process that materializes data as per the definition of a materialization task.

abstract error() Optional[BaseException][source]
abstract job_id() str[source]
abstract should_be_retried() bool[source]
abstract status() feast.infra.materialization.batch_materialization_engine.MaterializationJobStatus[source]
task: feast.infra.materialization.batch_materialization_engine.MaterializationTask
abstract url() Optional[str][source]
class feast.infra.materialization.MaterializationTask(project: str, feature_view: Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView], start_time: datetime.datetime, end_time: datetime.datetime, tqdm_builder: Callable[[int], tqdm.std.tqdm])[source]

Bases: object

A MaterializationTask represents a unit of data that needs to be materialized from an offline store to an online store.

end_time: datetime.datetime
feature_view: Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]
project: str
start_time: datetime.datetime
tqdm_builder: Callable[[int], tqdm.std.tqdm]