feast.infra.materialization package
Subpackages
Submodules
feast.infra.materialization.batch_materialization_engine module
- class feast.infra.materialization.batch_materialization_engine.BatchMaterializationEngine(*, repo_config: feast.repo_config.RepoConfig, offline_store: feast.infra.offline_stores.offline_store.OfflineStore, online_store: feast.infra.online_stores.online_store.OnlineStore, **kwargs)[source]
Bases:
abc.ABC
- abstract materialize(registry: feast.registry.BaseRegistry, tasks: List[feast.infra.materialization.batch_materialization_engine.MaterializationTask]) List[feast.infra.materialization.batch_materialization_engine.MaterializationJob] [source]
Materialize data from the offline store to the online store for this feature repo. :param registry: The feast registry containing the applied feature views. :param tasks: A list of individual materialization tasks.
- Returns
A list of materialization jobs representing each task.
- abstract teardown_infra(project: str, fvs: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities: Sequence[feast.entity.Entity])[source]
This method ensures that any infrastructure or resources set up by ``update()``are torn down.
- abstract update(project: str, views_to_delete: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], views_to_keep: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities_to_delete: Sequence[feast.entity.Entity], entities_to_keep: Sequence[feast.entity.Entity])[source]
This method ensures that any necessary infrastructure or resources needed by the engine are set up ahead of materialization.
- class feast.infra.materialization.batch_materialization_engine.MaterializationJob[source]
Bases:
abc.ABC
MaterializationJob represents an ongoing or executed process that materializes data as per the definition of a materialization task.
- abstract error() Optional[BaseException] [source]
- class feast.infra.materialization.batch_materialization_engine.MaterializationJobStatus(value)[source]
Bases:
enum.Enum
An enumeration.
- AVAILABLE = 3
- CANCELLED = 6
- CANCELLING = 5
- ERROR = 4
- RUNNING = 2
- SUCCEEDED = 7
- WAITING = 1
- class feast.infra.materialization.batch_materialization_engine.MaterializationTask(project: str, feature_view: Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView], start_time: datetime.datetime, end_time: datetime.datetime, tqdm_builder: Callable[[int], tqdm.std.tqdm])[source]
Bases:
object
A MaterializationTask represents a unit of data that needs to be materialized from an offline store to an online store.
- end_time: datetime.datetime
- feature_view: Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]
- start_time: datetime.datetime
feast.infra.materialization.local_engine module
- class feast.infra.materialization.local_engine.LocalMaterializationEngine(*, repo_config: feast.repo_config.RepoConfig, offline_store: feast.infra.offline_stores.offline_store.OfflineStore, online_store: feast.infra.online_stores.online_store.OnlineStore, **kwargs)[source]
Bases:
feast.infra.materialization.batch_materialization_engine.BatchMaterializationEngine
- materialize(registry, tasks: List[feast.infra.materialization.batch_materialization_engine.MaterializationTask]) List[feast.infra.materialization.batch_materialization_engine.MaterializationJob] [source]
Materialize data from the offline store to the online store for this feature repo. :param registry: The feast registry containing the applied feature views. :param tasks: A list of individual materialization tasks.
- Returns
A list of materialization jobs representing each task.
- teardown_infra(project: str, fvs: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities: Sequence[feast.entity.Entity])[source]
This method ensures that any infrastructure or resources set up by ``update()``are torn down.
- update(project: str, views_to_delete: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], views_to_keep: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities_to_delete: Sequence[feast.entity.Entity], entities_to_keep: Sequence[feast.entity.Entity])[source]
This method ensures that any necessary infrastructure or resources needed by the engine are set up ahead of materialization.
- class feast.infra.materialization.local_engine.LocalMaterializationEngineConfig(*, type: Literal['local'] = 'local')[source]
Bases:
feast.repo_config.FeastConfigBaseModel
Batch Materialization Engine config for local in-process engine
- type: Literal['local']
Type selector
- class feast.infra.materialization.local_engine.LocalMaterializationJob(job_id: str, status: feast.infra.materialization.batch_materialization_engine.MaterializationJobStatus, error: Union[BaseException, NoneType] = None)[source]
Bases:
feast.infra.materialization.batch_materialization_engine.MaterializationJob
- error() Optional[BaseException] [source]
Module contents
- class feast.infra.materialization.BatchMaterializationEngine(*, repo_config: feast.repo_config.RepoConfig, offline_store: feast.infra.offline_stores.offline_store.OfflineStore, online_store: feast.infra.online_stores.online_store.OnlineStore, **kwargs)[source]
Bases:
abc.ABC
- abstract materialize(registry: feast.registry.BaseRegistry, tasks: List[feast.infra.materialization.batch_materialization_engine.MaterializationTask]) List[feast.infra.materialization.batch_materialization_engine.MaterializationJob] [source]
Materialize data from the offline store to the online store for this feature repo. :param registry: The feast registry containing the applied feature views. :param tasks: A list of individual materialization tasks.
- Returns
A list of materialization jobs representing each task.
- abstract teardown_infra(project: str, fvs: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities: Sequence[feast.entity.Entity])[source]
This method ensures that any infrastructure or resources set up by ``update()``are torn down.
- abstract update(project: str, views_to_delete: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], views_to_keep: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities_to_delete: Sequence[feast.entity.Entity], entities_to_keep: Sequence[feast.entity.Entity])[source]
This method ensures that any necessary infrastructure or resources needed by the engine are set up ahead of materialization.
- class feast.infra.materialization.LocalMaterializationEngine(*, repo_config: feast.repo_config.RepoConfig, offline_store: feast.infra.offline_stores.offline_store.OfflineStore, online_store: feast.infra.online_stores.online_store.OnlineStore, **kwargs)[source]
Bases:
feast.infra.materialization.batch_materialization_engine.BatchMaterializationEngine
- materialize(registry, tasks: List[feast.infra.materialization.batch_materialization_engine.MaterializationTask]) List[feast.infra.materialization.batch_materialization_engine.MaterializationJob] [source]
Materialize data from the offline store to the online store for this feature repo. :param registry: The feast registry containing the applied feature views. :param tasks: A list of individual materialization tasks.
- Returns
A list of materialization jobs representing each task.
- teardown_infra(project: str, fvs: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities: Sequence[feast.entity.Entity])[source]
This method ensures that any infrastructure or resources set up by ``update()``are torn down.
- update(project: str, views_to_delete: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], views_to_keep: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities_to_delete: Sequence[feast.entity.Entity], entities_to_keep: Sequence[feast.entity.Entity])[source]
This method ensures that any necessary infrastructure or resources needed by the engine are set up ahead of materialization.
- class feast.infra.materialization.LocalMaterializationJob(job_id: str, status: feast.infra.materialization.batch_materialization_engine.MaterializationJobStatus, error: Union[BaseException, NoneType] = None)[source]
Bases:
feast.infra.materialization.batch_materialization_engine.MaterializationJob
- error() Optional[BaseException] [source]
- class feast.infra.materialization.MaterializationJob[source]
Bases:
abc.ABC
MaterializationJob represents an ongoing or executed process that materializes data as per the definition of a materialization task.
- abstract error() Optional[BaseException] [source]
- class feast.infra.materialization.MaterializationTask(project: str, feature_view: Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView], start_time: datetime.datetime, end_time: datetime.datetime, tqdm_builder: Callable[[int], tqdm.std.tqdm])[source]
Bases:
object
A MaterializationTask represents a unit of data that needs to be materialized from an offline store to an online store.
- end_time: datetime.datetime
- feature_view: Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]
- start_time: datetime.datetime