Skip to content

Container-Level Quality Scoring

DO Calculating quality scores

DQ Including all eight score dimensions

With DQ enabled, container scores use all eight dimensions. Without DQ, only timeliness and volumetrics contribute, and the other six dimensions have zero effective weight.

A container (table, view, file, or other structured data asset or any aggregation of data assets such as assets that share a common tag) receives an overall quality score derived from its constituent field scores and additional container-specific metrics. Container scores then roll up into the datastore score.

Which Containers Get a Score?

A container receives a quality score only if it has both a completed profile and a completed scan. Containers that have not been both profiled and scanned display - in the UI and are completely excluded from the datastore aggregate: they do not participate in the calculation.

Container State UI Display In Datastore Aggregate?
Synced only (no profile, no scan) - No
Profiled only (no scan) - No
Scanned only (no profile) - No
Profiled + Scanned, no checks Numeric score (low) Yes
Profiled + Scanned, with checks Numeric score Yes

Common Misconception

Containers showing - are not scored as 0 and do not drag down your datastore score. They are completely excluded from the aggregate: neither in the numerator nor the denominator. However, a container that was profiled and scanned but has no quality checks will receive a low numeric score (because unmonitored dimensions like coverage and accuracy are penalized heavily) and will participate in the aggregate. See the low datastore score example for how this plays out.

How Container Scores Are Calculated

Your container's total Quality Score starts at a baseline of 70. Each of the eight data quality dimensions then adjusts this baseline:

  • Dimension aggregation:
    • Completeness: Weighted average of all field completeness scores
    • Coverage: Weighted average of field coverage scores, adjusted for how often each active check is asserted
    • Conformity: Weighted average of field conformity scores, adjusted for schema-level conformity checks
    • Consistency: Weighted average of field consistency scores, adjusted for profiling frequency
    • Precision: Weighted average of field precision scores
    • Accuracy: Weighted average of field accuracy scores
    • Timeliness: Calculated using the process described in Timeliness Dimension
    • Volumetrics: Calculated using the process described in Volumetrics Dimension
  • Multiplicative adjustment: Each dimension applies a multiplicative factor to the baseline. Dimensions with strong quality signals boost the score slightly, while dimensions with poor signals or missing data can reduce it significantly.

    score = baseline × f(coverage) × f(accuracy) × f(conformity) × f(precision)
                     × f(consistency) × f(completeness) × f(timeliness) × f(volumetrics)
    

    Each factor is bounded: no single dimension can take over the entire score. The system caps both the maximum penalty and the maximum boost per dimension.

  • Weight controls: Higher weights make dimensions more influential. Setting a weight to zero removes that dimension's effect entirely

  • Missing value handling: When a dimension cannot be measured (e.g., no checks of that type exist), the system applies a default assumption. For some dimensions this means "assumed fine" (no penalty), while for others it means "unverified" (score reduction). See How Dimensions Influence the Score for details.
  • Special case: If only one dimension is weighted, the Quality Score mirrors that dimension's rating directly
  • Final clipping: Result is always constrained between 0 and 100

How Dimensions Influence the Score

Each dimension has a bounded range of influence: it can penalize the score when quality is poor and boost it slightly when quality is strong. The system is calibrated so that some dimensions have much more impact than others, reflecting their importance to overall data trustworthiness. The table lists each dimension's default impact level at default weights.

A dimension does not pull the score down just by being below 100. Each dimension has its own break-even point: a score above it nudges the total up, and a score below it pulls the total down, by an amount that grows with the dimension's weight. That is why a dimension can show a fairly low score and still boost the total.

Dimension Impact When No Checks Exist
Coverage High Score drops significantly: you can't trust what you don't measure
Accuracy High Score drops significantly: unvalidated data is assumed unreliable
Consistency High Score drops significantly: without profiling history, stability is unknown
Conformity Low Slight boost: absence of format-specific checks doesn't imply bad data
Precision Low Slight boost: absence of granularity checks doesn't imply bad data
Completeness Medium Small penalty: completeness is measured directly from profiling, not checks
Timeliness Medium Slight boost: without freshness checks, the data is assumed to arrive mostly on schedule
Volumetrics Medium Slight boost: without volumetric checks, the data's volume is assumed mostly stable

Dimension Impact Is Not Anomaly Impact

The Quality Score panel labels each dimension with an impact of Low, Medium, or High: how strongly that dimension is moving the score right now, which can differ from the default level above. This is separate from the Impact level of an anomaly and the Importance level of a check. Neither of those changes the quality score: each dimension counts its anomalies, or the rows they flag, without weighting them by Impact level.

Why Coverage and Accuracy Have the Biggest Impact

Coverage and Accuracy are the two most influential dimensions because they represent fundamental aspects of data governance:

  • Coverage measures whether you are actively monitoring your data at all. A field with zero quality checks provides zero assurance: the score reflects this lack of observability.
  • Accuracy measures whether your data passes the checks you've defined. If no accuracy-relevant checks exist, the system cannot confirm data correctness, so confidence is low.

By contrast, Conformity and Precision address specific formatting and granularity concerns. The absence of these checks doesn't mean the data is wrong: it simply means those particular aspects aren't being monitored. The system gives you the benefit of the doubt for these dimensions.

Consistency is also heavily weighted because it detects data drift. Without sufficient profiling history, the system cannot confirm that your data is stable, which reduces confidence.

Why a 70-Point Baseline?

The 70-point baseline represents a neutral confidence starting point.

  • Dimensions then adjust the baseline downward when issues are found or upward when strong quality signals exist.
  • This calibration ensures that new containers without extensive checks or history begin from a reasonable midpoint rather than 0.

Timeliness Dimension

The Timeliness score gauges whether data is available according to its expected schedule.

How Timeliness is Calculated

  • Scale: 0 to 100 based on adherence to freshness requirements
  • Field level: Directly inherited from the container's timeliness score
  • Anomaly counting: Counts distinct anomalies from the relevant check types within the measurement period (cutoff date)
  • Scoring model: Scores start at 100 and decrease based on anomaly count
    • The first anomaly has the largest impact
    • Each additional anomaly has diminishing impact (exponential decay)
    • As anomaly count grows, the score approaches 0 but the marginal penalty per anomaly shrinks
  • Applicable rule types: Time distribution size, freshness constraints
    See Timeliness Rule Types for the full Timeliness rule type list.

Score Interpretation

  • 100: No timeliness anomalies detected
  • 60–80: A small number of anomalies detected: worth investigating
  • 40–60: Multiple anomalies indicating recurring timeliness issues
  • Below 40: Significant and frequent anomaly counts indicating serious issues
  • None/Null: No checks of this type configured (unmeasured)

Volumetrics Dimension

The Volumetrics score analyzes consistency in data size and shape over time.

Shared Scoring Formula

Timeliness and Volumetrics both use the same exponential penalty formula for anomaly counts. This consistency ensures comparable scoring behavior across dimensions, even though the anomalies being measured differ.

How Volumetrics is Calculated

  • Scale: 0 to 100 based on volumetric stability
  • Field level: Directly inherited from the container's volumetrics score
  • Anomaly counting: Counts distinct anomalies from the relevant check types within the measurement period (cutoff date)
  • Scoring model: Scores start at 100 and decrease based on anomaly count
    • The first anomaly has the largest impact
    • Each additional anomaly has diminishing impact (exponential decay)
    • As anomaly count grows, the score approaches 0 but the marginal penalty per anomaly shrinks
  • Applicable rule types: Row count size, partition size constraints
    See Volumetric Rule Types for the full Volumetric rule type list.

Examples

  • Container with consistent record counts per partition → Score ~100
  • Container showing unexpected spikes or drops in volume → Score ~75
  • Container with erratic or missing time distributions → Score ~50

Additional Container-Level Factors

Beyond the eight dimensions, containers incorporate:

  • Check assertion frequency: Coverage rewards checks that are asserted often and recently. Each active check is scored on how often scans asserted it during the decay period, and recent assertions count more than older ones. The container's coverage modifier is the average across its active checks. Daily or more frequent assertion of every check maximizes it.
    • A scan that asserts every active check counts toward all of them. Scanning everything on a regular schedule keeps coverage at its highest.
    • A scan that asserts only selected checks counts only toward those checks. Checks left out of your scans lower coverage over time.
    • A check that a scan could not evaluate does not count as asserted.
    • A newly created check is not penalized for scans that ran before it existed.
    • When a container has no active checks, the modifier falls back to how often the container is scanned.
  • Profiling frequency: Regular profiling ensures statistics remain current and boosts consistency scores. Weekly or more frequent profiling gives the maximum boost. The consistency score is capped at 100 after the modifier is applied.
  • Field tag weights: Field weights (derived from tags) are used when calculating weighted averages for container-level dimensions. Fields with higher-weight tags have more influence on their container's dimension scores.

Most Impactful Dimensions

While specific scoring weights can be customized, dimensions that typically most influence quality scores are:

  • Coverage: Asserting frequent, comprehensive quality checks is critical
  • Accuracy: Large volumes of anomalies severely impact scores
  • Consistency: Erratic or unstable data characteristics reduce confidence

See Also