Skip to content

Field Count

Definition

Asserts that there must be exactly a specified number of fields.

General Properties

Name Supported
Filter
Allows the targeting of specific data based on conditions
Coverage Customization
Allows adjusting the percentage of records that must meet the rule's conditions

Specific Properties

Specify the exact number of fields expected in the profile.

Name Description
Number of Fields
The exact number of fields that should be present in the profile.

Anomaly Types

Type Supported
Record
Flag inconsistencies at the row level
Shape
Flag inconsistencies in the overall patterns and distributions of a field

Example

Objective: Ensure that the ORDERS profile contains exactly 9 fields.

Sample Profile

Valid
FIELD_NAME FIELD_TYPE
O_ORDERKEY STRING
O_CUSTKEY STRING
O_ORDERSTATUS STRING
O_TOTALPRICE FLOAT
O_ORDERDATE DATE
O_ORDERPRIORITY STRING
O_CLERK STRING
O_SHIPPRIORITY STRING
O_COMMENT STRING
Invalid

count (8) less than expected (9)

FIELD_NAME FIELD_TYPE
O_ORDERKEY STRING
O_CUSTKEY STRING
O_ORDERSTATUS STRING
O_TOTALPRICE FLOAT
O_ORDERDATE DATE
O_ORDERPRIORITY STRING
O_CLERK STRING
O_COMMENT STRING

count (10) greater than expected (9)

FIELD_NAME FIELD_TYPE
O_ORDERKEY STRING
O_CUSTKEY STRING
O_ORDERSTATUS STRING
O_TOTALPRICE FLOAT
O_ORDERDATE DATE
O_ORDERPRIORITY STRING
O_CLERK STRING
O_COMMENT STRING
EXTRA_FIELD UNKOWN
{
    "description": "Ensure that the ORDERS profile contains exactly 9 fields",
    "coverage": 1,
    "properties": {
        "value": 9
    },
    "tags": [],
    "fields": null,
    "additional_metadata": {"key 1": "value 1", "key 2": "value 2"},
    "rule": "fieldCount",
    "container_id": {container_id},
    "template_id": {template_id},
    "filter": "1=1"
}

Anomaly Explanation

Among the presented sample profiles, the second one is missing a field, while the third one contains an extra field. Only the first profile has the correct number of fields, which is 9.

graph TD
A[Start] --> B[Retrieve Profile Fields]
B --> C{Does the profile have exactly 9 fields?}
C -->|Yes| D[End]
C -->|No| E[Mark as Anomalous]
E --> D
-- An illustrative SQL query related to the rule.
select
    table_name, count(column_name) as number_of_fields
from information_schema.columns 
where table_name = 'orders'
group by table_name
having count(column_name) <> 9

Potential Violation Messages

Shape Anomaly

In ORDERS, the field count is not 9.


Last update: June 14, 2024