feast.dqm.profilers package
Submodules
feast.dqm.profilers.ge_profiler module
- class feast.dqm.profilers.ge_profiler.GEProfile(expectation_suite: ExpectationSuite)[source]
Bases:
Profile
GEProfile is an implementation of abstract Profile for integration with Great Expectations. It executes validation by applying expectations from ExpectationSuite instance to a given dataset.
- expectation_suite: ExpectationSuite
- validate(df: DataFrame) GEValidationReport [source]
Validate provided dataframe against GE expectation suite. 1. Pandas dataframe is converted into PandasDataset (GE type) 2. Some fixes applied to the data to avoid crashes inside GE (see _prepare_dataset) 3. Each expectation from ExpectationSuite instance tested against resulting dataset
Return GEValidationReport, which parses great expectation’s schema into list of generic ValidationErrors.
- class feast.dqm.profilers.ge_profiler.GEProfiler(user_defined_profiler: Callable[[DataFrame], ExpectationSuite], with_feature_metadata: bool = False)[source]
Bases:
Profiler
GEProfiler is an implementation of abstract Profiler for integration with Great Expectations. It wraps around user defined profiler that should accept dataset (in a form of pandas dataframe) and return ExpectationSuite.
- analyze_dataset(df: DataFrame) Profile [source]
Generate GEProfile with ExpectationSuite (set of expectations) from a given pandas dataframe by applying user defined profiler.
Some fixes are also applied to the dataset (see _prepare_dataset function) to make it compatible with GE.
Return GEProfile
- classmethod from_proto(proto: GEValidationProfiler) GEProfiler [source]
- class feast.dqm.profilers.ge_profiler.GEValidationReport(validation_result: Dict[Any, Any])[source]
Bases:
ValidationReport
- property errors: List[ValidationError]
Return list of ValidationErrors if validation failed (is_success = false)
feast.dqm.profilers.profiler module
- class feast.dqm.profilers.profiler.Profile[source]
Bases:
object
- abstract validate(dataset: DataFrame) ValidationReport [source]
Run set of rules / expectations from current profile against given dataset.
Return ValidationReport
- class feast.dqm.profilers.profiler.Profiler[source]
Bases:
object
- class feast.dqm.profilers.profiler.ValidationError(check_name: str, column_name: str, check_config: Any | None = None, missing_count: int | None = None, missing_percent: float | None = None, observed_value: float | None = None, unexpected_count: int | None = None, unexpected_percent: float | None = None)[source]
Bases:
object