feast.infra.materialization package

Subpackages

Submodules

feast.infra.materialization.batch_materialization_engine module

class feast.infra.materialization.batch_materialization_engine.BatchMaterializationEngine(*, repo_config: RepoConfig, offline_store: OfflineStore, online_store: OnlineStore, **kwargs)[source]

Bases: ABC

The interface that Feast uses to control the compute system that handles batch materialization.

abstract materialize(registry: BaseRegistry, tasks: List[MaterializationTask]) List[MaterializationJob][source]

Materialize data from the offline store to the online store for this feature repo.

Parameters:
  • registry – The registry for the current feature store.

  • tasks – A list of individual materialization tasks.

Returns:

A list of materialization jobs representing each task.

abstract teardown_infra(project: str, fvs: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], entities: Sequence[Entity])[source]

Tears down all cloud resources used by the materialization engine for the specified set of Feast objects.

Parameters:
  • project – Feast project to which the objects belong.

  • fvs – Feature views whose corresponding infrastructure should be deleted.

  • entities – Entities whose corresponding infrastructure should be deleted.

abstract update(project: str, views_to_delete: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], views_to_keep: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], entities_to_delete: Sequence[Entity], entities_to_keep: Sequence[Entity])[source]

Prepares cloud resources required for batch materialization for the specified set of Feast objects.

Parameters:
  • project – Feast project to which the objects belong.

  • views_to_delete – Feature views whose corresponding infrastructure should be deleted.

  • views_to_keep – Feature views whose corresponding infrastructure should not be deleted, and may need to be updated.

  • entities_to_delete – Entities whose corresponding infrastructure should be deleted.

  • entities_to_keep – Entities whose corresponding infrastructure should not be deleted, and may need to be updated.

class feast.infra.materialization.batch_materialization_engine.MaterializationJob[source]

Bases: ABC

A MaterializationJob represents an ongoing or executed process that materializes data as per the definition of a materialization task.

abstract error() BaseException | None[source]
abstract job_id() str[source]
abstract should_be_retried() bool[source]
abstract status() MaterializationJobStatus[source]
task: MaterializationTask
abstract url() str | None[source]
class feast.infra.materialization.batch_materialization_engine.MaterializationJobStatus(value)[source]

Bases: Enum

An enumeration.

AVAILABLE = 3
CANCELLED = 6
CANCELLING = 5
ERROR = 4
RUNNING = 2
SUCCEEDED = 7
WAITING = 1
class feast.infra.materialization.batch_materialization_engine.MaterializationTask(project: str, feature_view: BatchFeatureView | StreamFeatureView | FeatureView, start_time: datetime, end_time: datetime, tqdm_builder: Callable[[int], tqdm])[source]

Bases: object

A MaterializationTask represents a unit of data that needs to be materialized from an offline store to an online store.

end_time: datetime
feature_view: BatchFeatureView | StreamFeatureView | FeatureView
project: str
start_time: datetime
tqdm_builder: Callable[[int], tqdm]

feast.infra.materialization.local_engine module

class feast.infra.materialization.local_engine.LocalMaterializationEngine(*, repo_config: RepoConfig, offline_store: OfflineStore, online_store: OnlineStore, **kwargs)[source]

Bases: BatchMaterializationEngine

materialize(registry, tasks: List[MaterializationTask]) List[MaterializationJob][source]

Materialize data from the offline store to the online store for this feature repo.

Parameters:
  • registry – The registry for the current feature store.

  • tasks – A list of individual materialization tasks.

Returns:

A list of materialization jobs representing each task.

teardown_infra(project: str, fvs: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], entities: Sequence[Entity])[source]

Tears down all cloud resources used by the materialization engine for the specified set of Feast objects.

Parameters:
  • project – Feast project to which the objects belong.

  • fvs – Feature views whose corresponding infrastructure should be deleted.

  • entities – Entities whose corresponding infrastructure should be deleted.

update(project: str, views_to_delete: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], views_to_keep: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], entities_to_delete: Sequence[Entity], entities_to_keep: Sequence[Entity])[source]

Prepares cloud resources required for batch materialization for the specified set of Feast objects.

Parameters:
  • project – Feast project to which the objects belong.

  • views_to_delete – Feature views whose corresponding infrastructure should be deleted.

  • views_to_keep – Feature views whose corresponding infrastructure should not be deleted, and may need to be updated.

  • entities_to_delete – Entities whose corresponding infrastructure should be deleted.

  • entities_to_keep – Entities whose corresponding infrastructure should not be deleted, and may need to be updated.

class feast.infra.materialization.local_engine.LocalMaterializationEngineConfig(*, type: Literal['local'] = 'local')[source]

Bases: FeastConfigBaseModel

Batch Materialization Engine config for local in-process engine

type: Literal['local']

Type selector

class feast.infra.materialization.local_engine.LocalMaterializationJob(job_id: str, status: feast.infra.materialization.batch_materialization_engine.MaterializationJobStatus, error: BaseException | NoneType = None)[source]

Bases: MaterializationJob

error() BaseException | None[source]
job_id() str[source]
should_be_retried() bool[source]
status() MaterializationJobStatus[source]
task: MaterializationTask
url() str | None[source]

feast.infra.materialization.snowflake_engine module

class feast.infra.materialization.snowflake_engine.SnowflakeMaterializationEngine(*, repo_config: RepoConfig, offline_store: OfflineStore, online_store: OnlineStore, **kwargs)[source]

Bases: BatchMaterializationEngine

generate_snowflake_materialization_query(repo_config: RepoConfig, fv_latest_mapped_values_sql: str, feature_view: BatchFeatureView | FeatureView, feature_batch: list, project: str) str[source]
materialize(registry, tasks: List[MaterializationTask]) List[MaterializationJob][source]

Materialize data from the offline store to the online store for this feature repo.

Parameters:
  • registry – The registry for the current feature store.

  • tasks – A list of individual materialization tasks.

Returns:

A list of materialization jobs representing each task.

materialize_to_external_online_store(repo_config: RepoConfig, materialization_sql: str, feature_view: StreamFeatureView | FeatureView, pbar: tqdm) None[source]
materialize_to_snowflake_online_store(repo_config: RepoConfig, materialization_sql: str, feature_view: BatchFeatureView | FeatureView, project: str) None[source]
teardown_infra(project: str, fvs: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], entities: Sequence[Entity])[source]

Tears down all cloud resources used by the materialization engine for the specified set of Feast objects.

Parameters:
  • project – Feast project to which the objects belong.

  • fvs – Feature views whose corresponding infrastructure should be deleted.

  • entities – Entities whose corresponding infrastructure should be deleted.

update(project: str, views_to_delete: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], views_to_keep: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], entities_to_delete: Sequence[Entity], entities_to_keep: Sequence[Entity])[source]

Prepares cloud resources required for batch materialization for the specified set of Feast objects.

Parameters:
  • project – Feast project to which the objects belong.

  • views_to_delete – Feature views whose corresponding infrastructure should be deleted.

  • views_to_keep – Feature views whose corresponding infrastructure should not be deleted, and may need to be updated.

  • entities_to_delete – Entities whose corresponding infrastructure should be deleted.

  • entities_to_keep – Entities whose corresponding infrastructure should not be deleted, and may need to be updated.

class feast.infra.materialization.snowflake_engine.SnowflakeMaterializationEngineConfig(*, type: Literal['snowflake.engine'] = 'snowflake.engine', config_path: str | None = '/home/docs/.snowsql/config', account: str | None = None, user: str | None = None, password: str | None = None, role: str | None = None, warehouse: str | None = None, authenticator: str | None = None, database: StrictStr, schema: str | None = 'PUBLIC')[source]

Bases: FeastConfigBaseModel

Batch Materialization Engine config for Snowflake Snowpark Python UDFs

class Config[source]

Bases: object

allow_population_by_field_name = True
account: str | None

Snowflake deployment identifier – drop .snowflakecomputing.com

authenticator: str | None

Snowflake authenticator name

config_path: str | None

Snowflake config path – absolute path required (Cant use ~)

database: StrictStr

Snowflake database name

password: str | None

Snowflake password

role: str | None

Snowflake role name

schema_: str | None

Snowflake schema name

type: Literal['snowflake.engine']

Type selector

user: str | None

Snowflake user name

warehouse: str | None

Snowflake warehouse name

class feast.infra.materialization.snowflake_engine.SnowflakeMaterializationJob(job_id: str, status: feast.infra.materialization.batch_materialization_engine.MaterializationJobStatus, error: BaseException | NoneType = None)[source]

Bases: MaterializationJob

error() BaseException | None[source]
job_id() str[source]
should_be_retried() bool[source]
status() MaterializationJobStatus[source]
task: MaterializationTask
url() str | None[source]

Module contents