Getting Started with Source Datastores

What is a Source Datastore?

In Qualytics, a Source Datastore is where your data lives. It is the connection Qualytics opens to look at the tables or files inside a system so it can profile them, run quality checks, and detect anomalies.

You may also encounter this concept under names like data source or data connection. In Qualytics, we always call it a Source Datastore, and every underlying technology is treated the same way: whether the system is a relational database, a data warehouse, or a cloud object storage bucket, a Source Datastore is the parent that owns a collection of containers (tables, files, or other data groupings).

Everything else in the platform starts from the Source Datastore: once it exists, you can Sync it to discover the containers inside, Profile those containers to understand their contents, and Scan them for anomalies.

Datastore Lifecycle

Every datastore in Qualytics follows a structured lifecycle, from initial creation through ongoing data quality operations.

graph TD
    A["<b>Create</b><br/>Add datastore & test connection"] --> B["<b>Link Enrichment</b><br/>Choose the destination"]
    B --> C["<b>Sync</b><br/>Discover tables, files & fields"]
    C --> D["<b>Profile</b><br/>Analyze data & infer checks"]
    D --> E["<b>Scan</b><br/>Run checks & detect anomalies"]
    E -->|"Repeat"| C

Guided First Runs

A newly created datastore shows a Get started in three steps strip at the top of its overview page, with Sync, Profile, and Scan as connected step cards. Each step unlocks when the previous one completes, and its card reflects the operation's live status: waiting as To do, then Queued, Running, and finally Completed, or Failed with a Retry button. Users who can run operations launch each step directly from its card. Once all three operations have completed successfully at least once, the datastore is fully initialized and the strip no longer appears.

Architecture

The Qualytics Datastore framework is organized in four layers:

Supported Storage Engines

How It All Connects

graph LR
    IO["<b>Data access</b><br/>Read fields and records<br/>Write results to a linked enrichment destination"]

    IO --> NS1["<b>Tables in a Schema</b>"]
    IO --> NS2["<b>Files in a Folder</b>"]

    NS1 --> SC1["<b>RDBMS DDL</b>"]
    NS2 --> SC2["<b>AVRO, Parquet, CSV, JSON</b>"]

    SC1 --> SE1["Oracle DB"]
    SC1 --> SE2["PostgreSQL"]
    SC1 --> SE3["MySQL"]
    SC1 --> SE4["Snowflake"]
    SC1 --> SE5["Redshift"]
    SC1 --> SE6["...and more"]

    SC2 --> SE7["Amazon S3"]
    SC2 --> SE8["Azure Storage"]
    SC2 --> SE9["GCS"]

The key insight is that Qualytics treats all datastores uniformly at the data-access layer. Whether you connect a PostgreSQL database or an S3 bucket, Qualytics reads and writes Fields and Records the same way, enabling consistent profiling, scanning, and anomaly detection across your entire data landscape.

Available Connectors

Qualytics supports 20 JDBC relational database connectors and 3 DFS cloud storage connectors out of the box.

Deep Dive

Explore the details of each datastore type: how JDBC and DFS connectors work, their configuration, and supported features.

Connection

Learn how to set up and manage connections to your datastores: create new connections from scratch or reuse existing credentials.

Multiple-Schema

Discover and onboard multiple schemas from a single connection at once, including schema discovery, name templates, supported connectors, and API reference.

How-tos

Add, edit, and delete datastores, whether creating from scratch with a new connection or reusing an existing one.

Data Catalog Links

View and correct how the datastore links to assets in connected data catalogs.