Skip to content

Azure Data Lake Storage Troubleshooting

Connection failures from Azure Data Lake Storage fall into four categories: connectivity errors (the storage account host is unreachable), authentication errors (Azure rejected the account key or the service principal credentials), authorization errors (the identity is valid but lacks the right role on the account or container), and container errors (the filesystem or root path does not exist). The shape of the error message indicates which layer failed and where to look first.

Common Errors

Error Likely Cause Fix
Host is unreachable or DNS resolution failure on <account>.dfs.core.windows.net The storage account name is misspelled, the account was deleted, or a firewall, private endpoint, or network rule is blocking the dataplane. Confirm the account name in the Azure Portal > Storage accounts list. If the account uses a private endpoint or firewall rules, allow the dataplane network range.
Authentication failed. Server failed to authenticate the request. (Shared Key) The access key is incorrect, has extra whitespace from copy-paste, or was rotated in Azure and not updated in Qualytics. Re-copy the access key from Storage account > Security + networking > Access keys. Update the connection with the current value.
Status code: 401 or invalid_client (Service Principal) The Client ID, Client Secret, or Tenant ID is incorrect, or the client secret has expired. Verify all three values in Microsoft Entra ID > App registrations. Generate a new client secret if the current one has expired.
tenant_id is required in parameters for SERVICE_PRINCIPAL auth The connection is set to Service Principal authentication but the Tenant ID field is empty. Open the connection and fill in the Tenant ID from Microsoft Entra ID > Overview.
Status code: 403 or AuthorizationPermissionMismatch The identity authenticated successfully but is missing a role assignment on the storage account or container. Grant the identity the role listed in the Source or Enrichment tables, scoped to the account or container.
Container not found or The specified filesystem does not exist The container or filesystem name in the URI does not exist on the storage account, or the root path includes a typo. Verify the container name in Storage account > Containers. Confirm the URI follows the format abfss://<container>@<account>.dfs.core.windows.net/<path>.

Note

Less common Azure errors that do not match one of the patterns above are shown as the original message from the storage service. The patterns above cover the connectivity, authentication, authorization, and container cases reported most frequently.

Detailed Notes

Connectivity errors

If the message describes an unreachable host, a DNS failure, or a network timeout, the dataplane could not reach the storage account endpoint at all. This happens before Azure evaluates any credentials, so adjusting keys or roles will not help. Focus on the network path:

  • Confirm the storage account name in the URI matches an existing account in the Azure Portal > Storage accounts list.
  • If the account uses a private endpoint, confirm the dataplane network can resolve the private DNS zone (privatelink.dfs.core.windows.net) and reach the private IP.
  • If the account has network rules under Networking > Firewalls and virtual networks, allow the dataplane outbound IP range or virtual network.
  • For Service Principal authentication, also confirm the dataplane can reach login.microsoftonline.com. If outbound traffic to that host is blocked, the OAuth token request will fail before the storage account is contacted.

Authentication errors

If the message describes an invalid key, a failed signature, an invalid_client response, or an expired secret, Azure rejected the request at the authentication layer. Authorization roles are not evaluated at this stage, so adjusting role assignments will not help.

Shared Key authentication:

  • Re-copy the access key from Storage account > Security + networking > Access keys, watching for stray whitespace, trailing newlines, or truncated characters.
  • Confirm the access key belongs to the storage account named in the URI.
  • If access keys were recently rotated in Azure, update the connection with the new value. The previous key stops working immediately after rotation.

Service Principal authentication:

  • Verify the Client ID and Tenant ID in Microsoft Entra ID > App registrations. The Tenant ID also appears under Microsoft Entra ID > Overview.
  • Generate a new client secret under App registrations > Certificates & secrets if the current one has expired. Client secrets have a configurable expiration date and stop working at that time.
  • Confirm the Client Secret value (not the Secret ID) was copied. Azure shows the secret only once at creation time.
  • If the Tenant ID field is empty, the connection cannot be created. The validation message is tenant_id is required in parameters for SERVICE_PRINCIPAL auth.

Note

Authentication errors occur before any role assignment is evaluated. Adding roles will not resolve them. Only correcting the credentials does.

Authorization errors

If the credentials are valid but Azure returns a 403 or AuthorizationPermissionMismatch, the identity authenticated but lacks the required role on the storage account or container.

  • Missing role on the container or account: The identity needs the role listed in the Source or Enrichment tables. Source datastores require read access; enrichment datastores require read and write access.
  • Role scoped too narrowly: A role assigned to a single blob or a parent management group instead of the storage account or container will not apply. Assign the role at the container or storage account scope.
  • Role assignment still propagating: New role assignments can take a few minutes to take effect. If the role was just granted, retry after a short wait before changing anything else.

Use the Azure Portal > Storage account > Access Control (IAM) > Role assignments view to confirm which roles the identity has and at which scope.

Container errors

These errors point to the container or path itself, not your credentials or roles:

  • Container does not exist: The container name in the URI was misspelled or the container was deleted. Azure surfaces this as Container not found or The specified filesystem does not exist.
  • Root path typo: The container exists but the path after the container name does not. Confirm the URI follows the format abfss://<container>@<account>.dfs.core.windows.net/<path> and that the path matches an existing directory.
  • Hierarchical namespace not enabled: Azure Data Lake Storage requires the storage account to have hierarchical namespace enabled. Standard blob storage accounts without this option will not work. Confirm the setting under Storage account > Configuration > Hierarchical namespace.

Diagnostic order

Investigate in the order Azure evaluates requests: connectivity first, authentication second, authorization third, then container configuration. Resolving connectivity issues before credential changes avoids rotating keys that were never the problem.

Symptom Layer Where to look
Host unreachable, DNS failure, timeout, or failure to reach login.microsoftonline.com Connectivity Confirm the storage account name, firewall rules, private endpoint DNS, and outbound access from the dataplane
Authentication failed, Status code: 401, invalid_client, or missing Tenant ID Authentication Re-copy the access key, or verify the Client ID, Client Secret, and Tenant ID; role assignments are not evaluated at this stage
Status code: 403, AuthorizationPermissionMismatch Authorization Compare the identity's role assignments to the minimum-permissions tables and confirm the role is scoped to the container or storage account
Container not found, The specified filesystem does not exist Container configuration Confirm the container name, URI format, and that hierarchical namespace is enabled on the account

For the full credential setup, see Permissions and the connection-form fields on the Azure Data Lake Storage Connector page.