Skip to content

Azure Data Lake Storage Permissions

Qualytics calls the Azure Data Lake Storage Gen2 APIs with the credentials supplied in the connection form, reading from the storage account and (for enrichment) writing back to it. The identity behind the connection must have the permissions listed below.

Which identity carries those permissions depends on the authentication mode:

Authentication mode Identity that carries the storage permissions
Shared Key The storage account itself. The shared key grants full read and write access at the account level, so no extra Azure RBAC roles are needed.
Service Principal The Microsoft Entra ID application registration whose client ID and client secret are entered in the connection form. The Azure RBAC roles below must be assigned to this service principal.

Choosing between auth modes

Shared Key is the fastest path to a working connection, but the key cannot be scoped to a subset of containers or paths and rotating it requires updating every consumer. Service Principal supports fine-grained scoping through Azure RBAC and integrates with Microsoft Entra ID lifecycle controls. For long-lived production connections, Service Principal is the recommended option.

Minimum Permissions (Source Datastore)

A read-only assignment lets Qualytics discover, profile, and scan files in the storage account. It cannot write anywhere.

Shared Key

No additional Azure RBAC role assignment is required. The shared access key carries full read and write privileges on every container in the storage account. To keep the connection read-only in practice, point the Root Path at the directory Qualytics should scan and avoid configuring this datastore as an enrichment target.

Shared Key cannot be made read-only

A storage account key always grants read and write at the account level. If you need a credential that is strictly read-only, use Service Principal authentication with the Storage Blob Data Reader role.

Service Principal

Assign the following built-in Azure RBAC role to the service principal:

Role Scope Purpose
Storage Blob Data Reader Storage account, container, or path Read blob contents and list directories so Qualytics can discover, profile, and scan files

Scope the assignment at the narrowest level that still covers the Root Path configured in the connection. Azure RBAC supports assignment at the storage account, individual container, or specific directory path. See the Azure documentation on assigning RBAC roles for blob and queue data access.

Minimum Permissions (Enrichment Datastore)

An enrichment assignment needs read access plus write and delete, because Qualytics creates, updates, and removes the metadata files it produces.

Shared Key

No additional Azure RBAC role assignment is required. The shared access key already covers the read, write, and delete operations Qualytics performs when writing anomaly records, scan results, and metadata back to the storage account.

Service Principal

Assign the following built-in Azure RBAC role to the service principal:

Role Scope Purpose
Storage Blob Data Contributor Storage account, container, or path Read, write, and delete blobs so Qualytics can persist anomaly records, scan results, and metadata, and remove superseded enrichment files

Storage Blob Data Contributor is a superset of Storage Blob Data Reader, so you do not need to assign both. Scope the assignment to the container or directory that holds enrichment output.

Encrypted storage accounts (customer-managed keys)

If the storage account is encrypted with a customer-managed key in Azure Key Vault, the service principal must also have permission to use that key. Without the matching Key Vault access policy or Key Vault Crypto Service Encryption User role, blob reads and writes fail with an authorization error even when the Storage Blob Data role is assigned. This is enforced by Azure, not by Qualytics.

Where to find the storage account name

The URI field expects a container-level ABFS URI in the format abfss://<container>@<account>.dfs.core.windows.net. You compose it from your container name and the storage account name. The Azure Portal does not show the ABFS URI directly, but its Data Lake Storage endpoint confirms the <account> value.

Warning

Look at the Data Lake Storage primary endpoint (the URL ending in .dfs.core.windows.net), not the Blob service endpoint (.blob.core.windows.net). The connection form uses the DFS endpoint's account name.

  1. Open the Azure Portal and navigate to your storage account.
  2. In the left blade, select Endpoints.
  3. Read the value under Data Lake Storage > Primary endpoint. It has the form https://<account>.dfs.core.windows.net. The <account> portion is your storage account name.

In the Qualytics connection form, enter the ABFS URI in the URI field (e.g., abfss://raw@acmelake.dfs.core.windows.net) and use Root Path to point at the subfolder you care about (e.g., /landing/orders/). Do not include a file path in the URI itself.

In the API, the same ABFS URI is passed as connection.uri.

Next steps

  • Authentication

    Choose between Shared Key and Service Principal and learn how each credential is used.

    Authentication

  • Add a Source Datastore

    Step-by-step guide to register an Azure Data Lake Storage source datastore in Qualytics.

    Add Source