feast.dqm.profilers package

Submodules

feast.dqm.profilers.ge_profiler module

class feast.dqm.profilers.ge_profiler.GEProfile(expectation_suite: ExpectationSuite)[source]

Bases: Profile

GEProfile is an implementation of abstract Profile for integration with Great Expectations. It executes validation by applying expectations from ExpectationSuite instance to a given dataset.

expectation_suite: ExpectationSuite
classmethod from_proto(proto: GEValidationProfile) → GEProfile[source]
to_proto()[source]
validate(df: DataFrame) → GEValidationReport[source]

Validate provided dataframe against GE expectation suite. 1. Pandas dataframe is converted into PandasDataset (GE type) 2. Some fixes applied to the data to avoid crashes inside GE (see _prepare_dataset) 3. Each expectation from ExpectationSuite instance tested against resulting dataset

Return GEValidationReport, which parses great expectation’s schema into list of generic ValidationErrors.

class feast.dqm.profilers.ge_profiler.GEProfiler(user_defined_profiler: Callable[[DataFrame], ExpectationSuite], with_feature_metadata: bool = False)[source]

Bases: Profiler

GEProfiler is an implementation of abstract Profiler for integration with Great Expectations. It wraps around user defined profiler that should accept dataset (in a form of pandas dataframe) and return ExpectationSuite.

analyze_dataset(df: DataFrame) → Profile[source]

Generate GEProfile with ExpectationSuite (set of expectations) from a given pandas dataframe by applying user defined profiler.

Some fixes are also applied to the dataset (see _prepare_dataset function) to make it compatible with GE.

Return GEProfile

classmethod from_proto(proto: GEValidationProfiler) → GEProfiler[source]
to_proto()[source]
class feast.dqm.profilers.ge_profiler.GEValidationReport(validation_result: Dict[Any, Any])[source]

Bases: ValidationReport

property errors: List[ValidationError]

Return list of ValidationErrors if validation failed (is_success = false)

property is_success: bool

Return whether validation was successful

feast.dqm.profilers.ge_profiler.ge_profiler(*args, with_feature_metadata=False)[source]

feast.dqm.profilers.profiler module

class feast.dqm.profilers.profiler.Profile[source]

Bases: object

abstract classmethod from_proto(proto) → Profile[source]
abstract to_proto()[source]
abstract validate(dataset: DataFrame) → ValidationReport[source]

Run set of rules / expectations from current profile against given dataset.

Return ValidationReport

class feast.dqm.profilers.profiler.Profiler[source]

Bases: object

abstract analyze_dataset(dataset: DataFrame) → Profile[source]

Generate Profile object with dataset’s characteristics (with rules / expectations) from given dataset (as pandas dataframe).

abstract classmethod from_proto(proto) → Profiler[source]
abstract to_proto()[source]
class feast.dqm.profilers.profiler.ValidationError(check_name: str, column_name: str, check_config: Any | None = None, missing_count: int | None = None, missing_percent: float | None = None, observed_value: float | None = None, unexpected_count: int | None = None, unexpected_percent: float | None = None)[source]

Bases: object

check_config: Any | None
check_name: str
column_name: str
missing_count: int | None
missing_percent: float | None
observed_value: float | None
to_dict()[source]
unexpected_count: int | None
unexpected_percent: float | None
class feast.dqm.profilers.profiler.ValidationReport[source]

Bases: object

abstract property errors: List[ValidationError]

Return list of ValidationErrors if validation failed (is_success = false)

abstract property is_success: bool

Return whether validation was successful

Module contents