Fabric Analytics

Adding and configuring a Fabric Analytics connection within Qualytics empowers the platform to build a symbolic link with your schema to perform operations like data discovery, visualization, reporting, syncing, profiling, scanning, anomaly surveillance, and more.

This documentation provides a step-by-step guide on how to add Fabric Analytics as both a source and enrichment datastore in Qualytics. It covers the entire process, from initial connection setup to testing and finalizing the configuration.

By following these instructions, enterprises can ensure their Fabric Analytics environment is properly connected with Qualytics, unlocking the platform's potential to help you proactively manage your full data quality lifecycle.

Let's get started 🚀

Fabric Analytics Setup Guide

To connect Fabric Analytics with Qualytics, you need to configure a Service Principal in Microsoft Entra ID (Azure AD) and grant it access to your Fabric workspace. This section walks you through the required prerequisites and credential retrieval process.

Prerequisites

Before configuring the Fabric Analytics datastore in Qualytics, ensure the following requirements are met:

Step 1: Register an Application in Microsoft Entra ID

1. Log in to the Azure Portal and click on the App registrations (under Microsoft Entra ID).

2. Click on the + New registration to create and register a new application in Microsoft Entra ID.

3. Enter the required details:

4. Click on the Register button to create the application.

Step 2: Retrieve the Client ID and Tenant ID

1. After registration, you will be taken to the app's Overview page.

2. Copy the Application (client) ID: this is your Client ID.

3. Copy the Directory (tenant) ID: this is your Tenant ID.

Step 3: Create a Client Secret

1. In the app registration, go to Client secrets and click + New client secret.

2. After clicking + New client secret, enter a description and choose an expiration period.

3. Copy the Value of the newly created secret immediately: this is your Client Secret, and it will not be shown again.

Step 4: Enable Service Principal Access in Fabric

1. Sign in to Microsoft Fabric as a Fabric administrator, go to Settings, and click on the Admin portal.

2. Under Tenant settings, locate Service principals can call Fabric APIs, enable the setting, and specify the security group containing your service principal (or allow the entire organization).

Step 5: Grant Workspace Access to the Service Principal

1. In Microsoft Fabric, open your target Workspace and click Manage access.

2. Click + Add people or groups, search for your registered application (e.g., qualytics-fabric), and select it.

3. Assign the Contributor role (or higher) and click Add.

Datastore Fabric Analytics Privileges

Qualytics connects to Fabric Analytics through the Microsoft JDBC driver for SQL Server using the SQL analytics endpoint. It queries system views (sys.schemas, sys.database_principals) to discover schemas and uses standard JDBC metadata APIs for tables, columns, and primary keys.

Minimum Fabric Analytics Permissions (Source Datastore)

Example: Verifying Service Principal Access

After configuring the Service Principal, you can verify access using the Azure CLI:

# Verify the app registration exists
az ad app show --id <client_id>

# Verify the Service Principal has the Contributor role on the Fabric workspace
az role assignment list --assignee <client_id> --scope <workspace_resource_id>

Troubleshooting Common Errors

Detailed Troubleshooting Notes

Authentication Errors

The error Login failed for user indicates that the Service Principal credentials are incorrect or expired.

Common causes:

• Expired Client Secret: the Client Secret has a configurable expiration date. It may have expired since the connection was created.
• Wrong Tenant ID: the Tenant ID does not match the Microsoft Entra ID tenant where the app is registered.
• Wrong Client ID: the Client ID (Application ID) does not match the app registration.

Permission Errors

The error The SELECT permission was denied on object or Service principals are not allowed means the Service Principal authenticated successfully but lacks the necessary access.

Common causes:

• Workspace role insufficient: the Service Principal needs at least the Contributor role on the Fabric workspace.
• Tenant setting disabled: the "Service principals can use Fabric APIs" setting is not enabled in the Fabric Admin Portal.
• Security group restriction: the tenant setting is enabled but restricted to a specific security group that does not include the Service Principal.

Connection Errors

The error SQL analytics endpoint is not available or Cannot open database requested by the login indicates a configuration issue.

Common causes:

• SQL endpoint not enabled: the SQL analytics endpoint is not enabled for the Lakehouse or Warehouse.
• Wrong endpoint: the SQL analytics endpoint URL does not match the target Lakehouse/Warehouse.
• Fabric capacity paused: the Fabric capacity is paused, making the SQL endpoint unavailable.

Retrieve the SQL Analytics Endpoint

1. Open your Lakehouse or Warehouse in the Fabric workspace and copy the SQL analytics endpoint from the connection string area.

The endpoint format is:

<workspace-guid>.datawarehouse.fabric.microsoft.com

Add the Source Datastore

A source datastore is a storage location Qualytics connects to so it can profile, scan, and monitor data. Adding Fabric Analytics as a source lets Qualytics query your Lakehouse or Warehouse through its SQL analytics endpoint and run quality operations on the tables it discovers.

Before you start, complete the service principal setup in the Fabric Analytics Setup Guide above and have the SQL analytics endpoint at hand.

Field reference

The Add Datastore page shows the sections below when Fabric Analytics is selected. When reusing an existing connection, the Connection Properties and Secrets Management sections come already filled in and read-only: Qualytics has already validated those credentials, so you fill in only the Datastores Extraction and the Datastore Properties. To change a saved connection's credentials, edit the connection through the Manage Connections page; edits there apply to every datastore that reuses the connection.

Connection Properties

These fields define the Fabric endpoint Qualytics connects to and the service principal it connects with. They belong to the connection: when reusing an existing connection, they come already filled in and read-only.

Secrets Management

This group is optional: use it only if you want Qualytics to pull credentials from a secrets manager instead of typing them into the form. Turn on HashiCorp Vault to show the fields below. Despite the label, any secrets manager that exposes a compatible REST API works, not only HashiCorp Vault; see Secrets Management. It also belongs to the connection: read-only when reusing an existing connection.

Datastores Extraction

Pick the Lakehouse or Warehouse and the schema Qualytics should read from. You fill these in on both flows.

Datastore Properties

Common fields for every source datastore, shown below the Datastores Extraction section. You fill these in on both flows.

Steps

There are two ways to set up the connection: reuse a connection you already saved (Existing Connection) or create a new one from scratch (New Connection). The tabs below walk through each option; pick the one you want to follow. Each field is described in the Field reference above.

API Payload Examples

Creating a Source Datastore

This section provides a sample payload for creating a Fabric Analytics datastore. Replace the placeholder values with actual data relevant to your setup.

Endpoint (Post): /api/datastores (post)

{
    "name": "your_datastore_name",
    "teams": ["Public"],
    "database": "fabric_database",
    "schema": "dbo",
    "enrichment_only": false,
    "trigger_sync": true,
    "connection": {
        "name": "your_connection_name",
        "type": "fabric",
        "host": "your-workspace-guid.datawarehouse.fabric.microsoft.com",
        "port": 1433,
        "username": "your_client_id",
        "password": "your_client_secret",
        "parameters": {
            "tenant_id": "your_tenant_id"
        }
    }
}

{
    "name": "your_datastore_name",
    "teams": ["Public"],
    "database": "fabric_database",
    "schema": "dbo",
    "enrichment_only": false,
    "trigger_sync": true,
    "connection_id": 123
}

# Step 1: Create a Connection
qualytics connections create \
    --type fabric \
    --name "your_connection_name" \
    --host ${FABRIC_HOST} \
    --port 1433 \
    --username ${APPLICATION_CLIENT_ID} \
    --password ${CLIENT_SECRET} \
    --parameters '{"tenant_id": "azure-ad-tenant-id"}'

# Step 2: Create a Source Datastore
qualytics datastores create \
    --name "your_datastore_name" \
    --connection-name "your_connection_name" \
    --database your_database \
    --schema dbo