feast.infra.materialization.contrib.bytewax package

Submodules

feast.infra.materialization.contrib.bytewax.bytewax_materialization_engine

class feast.infra.materialization.contrib.bytewax.bytewax_materialization_engine.BytewaxMaterializationEngine(*, repo_config: RepoConfig, offline_store: OfflineStore, online_store: OnlineStore, **kwargs)[source]

Bases: BatchMaterializationEngine

materialize(registry: BaseRegistry, tasks: List[MaterializationTask]) → List[MaterializationJob][source]

Materialize data from the offline store to the online store for this feature repo.

Parameters:

registry – The registry for the current feature store.
tasks – A list of individual materialization tasks.

Returns:

A list of materialization jobs representing each task.

teardown_infra(project: str, fvs: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], entities: Sequence[Entity])[source]: This method ensures that any infrastructure or resources set up by ``update()``are torn down.

update(project: str, views_to_delete: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], views_to_keep: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], entities_to_delete: Sequence[Entity], entities_to_keep: Sequence[Entity])[source]: This method ensures that any necessary infrastructure or resources needed by the engine are set up ahead of materialization.

class feast.infra.materialization.contrib.bytewax.bytewax_materialization_engine.BytewaxMaterializationEngineConfig(*, type: Literal['bytewax'] = 'bytewax', namespace: StrictStr = 'default', image: StrictStr = 'bytewax/bytewax-feast:latest', env: List[dict] = [], image_pull_secrets: List[dict] = [], resources: dict = {}, service_account_name: StrictStr = '', annotations: dict = {})[source]

Bases: FeastConfigBaseModel

Batch Materialization Engine config for Bytewax

annotations: dict: (optional) Annotations to apply to the job container. Useful for linking the service account to IAM roles, operational metadata, etc

env: List[dict]: (optional) A list of environment variables to set in the created Kubernetes pods. These environment variables can be used to reference Kubernetes secrets.

image: StrictStr: (optional) The container image to use when running the materialization job.

image_pull_secrets: List[dict]: (optional) The secrets to use when pulling the image to run for the materialization job

namespace: StrictStr: (optional) The namespace in Kubernetes to use when creating services, configuration maps and jobs.

resources: dict: (optional) The resource requests and limits for the materialization containers

service_account_name: StrictStr: (optional) The service account name to use when running the job

type: Literal['bytewax']: Materialization type selector

feast.infra.materialization.contrib.bytewax.bytewax_materialization_job

class feast.infra.materialization.contrib.bytewax.bytewax_materialization_job.BytewaxMaterializationJob(job_id, namespace, error: BaseException | None = None)[source]

Bases: MaterializationJob

error()[source]

job_id()[source]

should_be_retried()[source]

status()[source]

task: MaterializationTask

url()[source]

Module contents

class feast.infra.materialization.contrib.bytewax.BytewaxMaterializationDataflow(config: RepoConfig, feature_view: FeatureView, paths: List[str])[source]

Bases: object

input_builder(worker_index, worker_count, _state)[source]

output_builder(worker_index, worker_count)[source]

process_path(path)[source]

class feast.infra.materialization.contrib.bytewax.BytewaxMaterializationEngine(*, repo_config: RepoConfig, offline_store: OfflineStore, online_store: OnlineStore, **kwargs)[source]

Bases: BatchMaterializationEngine

materialize(registry: BaseRegistry, tasks: List[MaterializationTask]) → List[MaterializationJob][source]

Materialize data from the offline store to the online store for this feature repo.

Parameters:

registry – The registry for the current feature store.
tasks – A list of individual materialization tasks.

Returns:

A list of materialization jobs representing each task.

teardown_infra(project: str, fvs: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], entities: Sequence[Entity])[source]: This method ensures that any infrastructure or resources set up by ``update()``are torn down.

update(project: str, views_to_delete: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], views_to_keep: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], entities_to_delete: Sequence[Entity], entities_to_keep: Sequence[Entity])[source]: This method ensures that any necessary infrastructure or resources needed by the engine are set up ahead of materialization.

class feast.infra.materialization.contrib.bytewax.BytewaxMaterializationEngineConfig(*, type: Literal['bytewax'] = 'bytewax', namespace: StrictStr = 'default', image: StrictStr = 'bytewax/bytewax-feast:latest', env: List[dict] = [], image_pull_secrets: List[dict] = [], resources: dict = {}, service_account_name: StrictStr = '', annotations: dict = {})[source]

Bases: FeastConfigBaseModel

Batch Materialization Engine config for Bytewax

annotations: dict: (optional) Annotations to apply to the job container. Useful for linking the service account to IAM roles, operational metadata, etc

env: List[dict]: (optional) A list of environment variables to set in the created Kubernetes pods. These environment variables can be used to reference Kubernetes secrets.

image: StrictStr: (optional) The container image to use when running the materialization job.

image_pull_secrets: List[dict]: (optional) The secrets to use when pulling the image to run for the materialization job

namespace: StrictStr: (optional) The namespace in Kubernetes to use when creating services, configuration maps and jobs.

resources: dict: (optional) The resource requests and limits for the materialization containers

service_account_name: StrictStr: (optional) The service account name to use when running the job

type: Literal['bytewax']: Materialization type selector

class feast.infra.materialization.contrib.bytewax.BytewaxMaterializationJob(job_id, namespace, error: BaseException | None = None)[source]

Bases: MaterializationJob

error()[source]

job_id()[source]

should_be_retried()[source]

status()[source]

task: MaterializationTask

url()[source]

class feast.infra.materialization.contrib.bytewax.BytewaxMaterializationTask(project, feature_view, start_date, end_date, tqdm)[source]

Bases: MaterializationTask

end_time: datetime

feature_view: BatchFeatureView | StreamFeatureView | FeatureView

project: str

start_time: datetime

tqdm_builder: Callable[[int], tqdm]