feast.infra.materialization.contrib.bytewax package
Submodules
feast.infra.materialization.contrib.bytewax.bytewax_materialization_engine
- class feast.infra.materialization.contrib.bytewax.bytewax_materialization_engine.BytewaxMaterializationEngine(*, repo_config: feast.repo_config.RepoConfig, offline_store: feast.infra.offline_stores.offline_store.OfflineStore, online_store: feast.infra.online_stores.online_store.OnlineStore, **kwargs)[source]
Bases:
feast.infra.materialization.batch_materialization_engine.BatchMaterializationEngine
- materialize(registry: feast.infra.registry.base_registry.BaseRegistry, tasks: List[feast.infra.materialization.batch_materialization_engine.MaterializationTask]) List[feast.infra.materialization.batch_materialization_engine.MaterializationJob] [source]
Materialize data from the offline store to the online store for this feature repo.
- Parameters
registry – The registry for the current feature store.
tasks – A list of individual materialization tasks.
- Returns
A list of materialization jobs representing each task.
- teardown_infra(project: str, fvs: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities: Sequence[feast.entity.Entity])[source]
This method ensures that any infrastructure or resources set up by ``update()``are torn down.
- update(project: str, views_to_delete: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], views_to_keep: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities_to_delete: Sequence[feast.entity.Entity], entities_to_keep: Sequence[feast.entity.Entity])[source]
This method ensures that any necessary infrastructure or resources needed by the engine are set up ahead of materialization.
- class feast.infra.materialization.contrib.bytewax.bytewax_materialization_engine.BytewaxMaterializationEngineConfig(*, type: Literal['bytewax'] = 'bytewax', namespace: pydantic.types.StrictStr = 'default', image: pydantic.types.StrictStr = 'bytewax/bytewax-feast:latest', env: List[dict] = [])[source]
Bases:
feast.repo_config.FeastConfigBaseModel
Batch Materialization Engine config for Bytewax
- env: List[dict]
(optional) A list of environment variables to set in the created Kubernetes pods. These environment variables can be used to reference Kubernetes secrets.
- image: pydantic.types.StrictStr
(optional) The container image to use when running the materialization job.
- namespace: pydantic.types.StrictStr
(optional) The namespace in Kubernetes to use when creating services, configuration maps and jobs.
- type: Literal['bytewax']
Materialization type selector
feast.infra.materialization.contrib.bytewax.bytewax_materialization_job
- class feast.infra.materialization.contrib.bytewax.bytewax_materialization_job.BytewaxMaterializationJob(job_id, namespace, error: Optional[BaseException] = None)[source]
Bases:
feast.infra.materialization.batch_materialization_engine.MaterializationJob
Module contents
- class feast.infra.materialization.contrib.bytewax.BytewaxMaterializationDataflow(config: feast.repo_config.RepoConfig, feature_view: feast.feature_view.FeatureView, paths: List[str])[source]
Bases:
object
- class feast.infra.materialization.contrib.bytewax.BytewaxMaterializationEngine(*, repo_config: feast.repo_config.RepoConfig, offline_store: feast.infra.offline_stores.offline_store.OfflineStore, online_store: feast.infra.online_stores.online_store.OnlineStore, **kwargs)[source]
Bases:
feast.infra.materialization.batch_materialization_engine.BatchMaterializationEngine
- materialize(registry: feast.infra.registry.base_registry.BaseRegistry, tasks: List[feast.infra.materialization.batch_materialization_engine.MaterializationTask]) List[feast.infra.materialization.batch_materialization_engine.MaterializationJob] [source]
Materialize data from the offline store to the online store for this feature repo.
- Parameters
registry – The registry for the current feature store.
tasks – A list of individual materialization tasks.
- Returns
A list of materialization jobs representing each task.
- teardown_infra(project: str, fvs: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities: Sequence[feast.entity.Entity])[source]
This method ensures that any infrastructure or resources set up by ``update()``are torn down.
- update(project: str, views_to_delete: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], views_to_keep: Sequence[Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]], entities_to_delete: Sequence[feast.entity.Entity], entities_to_keep: Sequence[feast.entity.Entity])[source]
This method ensures that any necessary infrastructure or resources needed by the engine are set up ahead of materialization.
- class feast.infra.materialization.contrib.bytewax.BytewaxMaterializationEngineConfig(*, type: Literal['bytewax'] = 'bytewax', namespace: pydantic.types.StrictStr = 'default', image: pydantic.types.StrictStr = 'bytewax/bytewax-feast:latest', env: List[dict] = [])[source]
Bases:
feast.repo_config.FeastConfigBaseModel
Batch Materialization Engine config for Bytewax
- env: List[dict]
(optional) A list of environment variables to set in the created Kubernetes pods. These environment variables can be used to reference Kubernetes secrets.
- image: pydantic.types.StrictStr
(optional) The container image to use when running the materialization job.
- namespace: pydantic.types.StrictStr
(optional) The namespace in Kubernetes to use when creating services, configuration maps and jobs.
- type: Literal['bytewax']
Materialization type selector
- class feast.infra.materialization.contrib.bytewax.BytewaxMaterializationJob(job_id, namespace, error: Optional[BaseException] = None)[source]
Bases:
feast.infra.materialization.batch_materialization_engine.MaterializationJob
- class feast.infra.materialization.contrib.bytewax.BytewaxMaterializationTask(project, feature_view, start_date, end_date, tqdm)[source]
Bases:
feast.infra.materialization.batch_materialization_engine.MaterializationTask
- end_time: datetime.datetime
- feature_view: Union[feast.batch_feature_view.BatchFeatureView, feast.stream_feature_view.StreamFeatureView, feast.feature_view.FeatureView]
- start_time: datetime.datetime