Contains Email

Think of Contains Email as a pattern check for contact data.

Any time your system stores customer or user information, email addresses are often the primary identifier used for communication, login, billing, and notifications. This check evaluates whether the selected field contains a recognizable email address.

Add Contains Email Check

Use the Contains Email check to evaluate whether a selected field includes a recognizable email pattern and to surface records that do not match its criteria.

What Does Contains Email Do?

Contains Email can surface values with issues such as:

• Missing @ or domain name
• Truncated or partially stored email values
• Corrupted email data during ingestion or transformation
• Non-email text accidentally stored in an email field

In simple terms: It checks for text that has a recognizable email structure. It does not confirm that the mailbox exists or can receive a message.

How Does Contains Email Work?

Step 1: Choose the Field

You select one field that is expected to contain an email address (for example: email, contact_details, or a JSON column containing email data).

Step 2: The Validation Happens

During a Scan, Qualytics evaluates each record in scope against the email pattern used by the rule.

Step 3: Get Your Results

• Pass: The field contains text that matches the email pattern.
• Anomaly Found: The value does not match the check criteria.

Why Should You Use Contains Email?

1. Prevent Broken Communication

If emails are malformed, automated messages can fail silently. Customers may not receive confirmations, alerts, or updates.

2. Improve Data Quality Upstream

Surfacing values that do not match the pattern can help teams investigate downstream issues in CRM systems, marketing tools, and analytics.

3. Avoid Operational Errors

Email values that do not match expected patterns can contribute to:

• Failed notifications
• Login issues
• Customer support escalations
• Exceptions to documented contact-data policies

Contains Email provides one quality signal for reviewing contact data.

Real-Life Example: Customer Contact Data Review

Let me walk you through a complete, real-world scenario:

The Situation

BrightCart is an online retail platform that collects customer contact details during account creation and checkout. Email addresses are critical because they are used for:

• Order confirmations
• Shipping notifications
• Password resets
• Customer support communication

Before sharing this data with marketing and support teams, the data team reviews whether the contact values meet its documented expectations.

The Problem They Had

Initially, the team relied on manual review to validate email data.

One week, customer support noticed an increase in complaints like:

• “I didn’t receive my order confirmation”
• “Password reset email never arrived”

When the data team investigated, they found that some customer records did not contain recognizable email addresses, but these issues had gone unnoticed during manual review.

After digging deeper, they discovered:

• The dataset had grown to tens of thousands of records
• Analysts were reviewing only a small sample due to time constraints
• Subtle issues (like missing domains or incomplete emails) were easy to miss
• Manual checks were inconsistent and error-prone

The Solution: Contains Email

To avoid relying only on visual inspection, the team introduced the Contains Email check for email-related fields during each scheduled Scan.

Instead of checking a small sample by hand, the rule applied the same criteria to the records included in the Scan.

What the System Discovered

ALERT GENERATED:

Technical Output (from Qualytics):

When reviewing the flagged records, the team noticed values like:

Summary

• Manual review did not scale as data volume increased
• Incorrect values slipped through unnoticed
• The check applied consistent criteria across the Scan scope
• The resulting anomalies gave the team a focused set of records to review

The Outcome

Immediate Benefits:

• The team corrected the rule configuration
• Email validation was applied to the correct data
• The team documented how the rule should be interpreted

Long-Term Benefits:

• Repeatable evaluation for records included in each Scan
• A smaller set of failed records for data stewards to investigate
• Shared context for marketing, support, and data teams

Key Takeaway

The Contains Email check evaluates each record included in a Scan consistently, avoiding the coverage gaps of manual sampling. Data stewards still review failed records and business context to decide whether a value is genuinely incorrect or an accepted exception.

Field Scope

Single: The rule evaluates a single specified field.

Accepted Types

General Properties

The filter allows you to define a subset of data upon which the rule will operate.

It requires a valid Spark SQL expression that determines the criteria rows in the DataFrame should meet. This means the expression specifies which rows the DataFrame should include based on those criteria. Since it's applied directly to the Spark DataFrame, traditional SQL constructs like WHERE clauses are not supported.

O_TOTALPRICE > 1000

C_MKTSEGMENT = 'BUILDING'

WHERE O_TOTALPRICE > 1000

WHERE C_MKTSEGMENT = 'BUILDING'

O_ORDERPRIORITY = '1-URGENT' AND O_ORDERSTATUS = 'O'

(L_SHIPDATE = '1998-09-02' OR L_RECEIPTDATE = '1998-09-01') AND L_RETURNFLAG = 'R'

WHERE O_ORDERPRIORITY = '1-URGENT' AND O_ORDERSTATUS = 'O'

O_TOTALPRICE > 1000, O_ORDERSTATUS = 'O'

RIGHT(
    O_ORDERPRIORITY,
    LENGTH(O_ORDERPRIORITY) - INSTR('-', O_ORDERPRIORITY)
) = 'URGENT'

LEVENSHTEIN(C_NAME, 'Supplier#000000001') < 7

RIGHT(
    O_ORDERPRIORITY,
    LENGTH(O_ORDERPRIORITY) - CHARINDEX('-', O_ORDERPRIORITY)
) = 'URGENT'

EDITDISTANCE(C_NAME, 'Supplier#000000001') < 7

O_ORDERSTATUS IN (
    SELECT DISTINCT O_ORDERSTATUS
    FROM {{_qualytics_self}}
    WHERE O_TOTALPRICE > 1000
)

O_ORDERSTATUS IN (
    SELECT DISTINCT O_ORDERSTATUS
    FROM ORDERS
    WHERE O_TOTALPRICE > 1000
)

Anomaly Types

Example

Objective: Check whether C_CONTACT_DETAILS contains at least one recognizable email pattern for records in the Scan scope.

Sample Data

{
    "description": "Check that C_CONTACT_DETAILS entries contain a recognizable email pattern",
    "coverage": 1,
    "properties": {},
    "tags": [],
    "fields": ["C_CONTACT_DETAILS"],
    "additional_metadata": {"key 1": "value 1", "key 2": "value 2"},
    "rule": "containsEmail",
    "container_id": {container_id},
    "template_id": {template_id},
    "filter": "1=1"
}

Anomaly Explanation

In the sample data above, the entry with C_CUSTKEY 2 does not satisfy the rule because its C_CONTACT_DETAILS value does not follow a typical email format.

graph TD
A[Start] --> B[Retrieve C_CONTACT_DETAILS]
B --> C{Does C_CONTACT_DETAILS contain an email address?}
C -->|Yes| D[Move to Next Record/End]
C -->|No| E[Mark as Anomalous]
E --> D

-- An illustrative SQL query demonstrating the rule applied to example dataset(s).
select
    c_custkey,
    c_contact_details
from customer 
where
    not regexp_like(c_contact_details, '^[a-zA-Z0-9._%-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,4}$')

Potential Violation Messages