Max Partition Size Anomaly Reporting

How the Max Partition Size check reports violations: the Shape Anomaly it produces, what the numbers mean, and how a partition-level anomaly differs from a row-level one.

Anomaly Messages

Max Partition Size always reports a Shape Anomaly, one per failing partition. There is no Record Anomaly: the rule evaluates a count, not individual rows, so there is no offending row to point at.

Shape Anomaly

There are more than <value> records present in <partition>, Some(<record_count>) were present

Reading the message:

<value> is the Maximum partition size configured on the check.
<partition> is the name of the partition that failed, which is the file name for a file container and the loaded slice for a table.
Some(<record_count>) is the measured record count for that partition. The count is reported wrapped in Some(...), so a partition of 622,880 records reads Some(622880).

When the message reads None

The measured count is not always available. When it is missing, the comparison falls back to the number of rows in the loaded data, but the message still prints the measured count, so it reads None were present instead of a number. The threshold and the partition name in the message are unaffected, and the anomaly is still correct: the partition did exceed the ceiling.

Because the anomaly is raised against the partition rather than against a set of rows, the Source Records view does not list violating records: there are none to list. What the anomaly gives you is the partition name and the two counts.

Custom Anomaly Description does not apply to this rule. The option replaces the message on Record Anomalies, and Max Partition Size emits a Shape Anomaly only.

Max Partition Size Anomaly Reporting

Anomaly Messages

See Also