# Azure Data Lake Storage Azure Data Lake Storage API

Source: https://userguide.qualytics.io/source-datastore/datastore/dfs/azure-datalake-storage/how-tos/api/

This page documents Azure Data Lake Storage Gen2-specific payload fields for the datastore API. For generic operations (get, list, update, delete, link enrichment), see [Datastore API](https://userguide.qualytics.io/source-datastore/datastore/api/).

> **Tip: Complete API Reference**
>
> For the full interactive API documentation with all request/response schemas, visit the [API docs](https://userguide.qualytics.io/api/docs).

Azure Data Lake Storage Gen2 supports two authentication types (**Shared Key** by default, or **Service Principal**) and can be created as a source datastore (read-only) or an enrichment datastore (read-write). All flows use the same endpoint:

**Endpoint**: `POST /api/datastores`

**Permission**: Manager, plus the **Editor** team permission on at least one of the assigned teams (the **Public** team when `teams` is left out). Admins are exempt.

---

## UI ↔ API field mapping

The API names a few fields differently from the UI. The mapping below avoids the most common copy-paste mistakes when switching from the form to the JSON payload.

| UI form field | API field | Notes | Example |
| --- | --- | --- | --- |
| **Connection > Connection Name** | `connection.name` | Saved-connection label. Optional. | `acme_adls_lake` |
| (set by connector choice) | `connection.type` | Connector identifier. Always `"abfs"` for Azure Data Lake Storage Gen2. | `abfs` |
| **Connection > URI** | `connection.uri` | Container-level ABFS URI. | `abfss://raw@acmelake.dfs.core.windows.net` |
| **Location > Root Path** | `root_path` | Subfolder inside the container. | `/raw/orders/` |
| **Connection > Type** | `connection.parameters.authentication_type` | `SHARED_KEY` (Shared Key) or `SERVICE_PRINCIPAL`. Defaults to `SHARED_KEY` when omitted. | `SHARED_KEY` |
| **Connection > Account Name** (Shared Key) / **Client ID** (Service Principal) | `connection.access_key` | UI label changes with `authentication_type`. The API field name stays `access_key`. | `acmelake` or `00000000-0000-0000-0000-000000000000` |
| **Connection > Access Key** (Shared Key) / **Client Secret** (Service Principal) | `connection.secret_key` | UI label changes with `authentication_type`. The API field name stays `secret_key`. | `<storage-account-access-key>` or `<client-secret>` |
| **Connection > Tenant ID** | `connection.parameters.tenant_id` | Required only when `authentication_type` is `SERVICE_PRINCIPAL`. | `11111111-1111-1111-1111-111111111111` |
| **Datastore > Initiate Sync** | `trigger_sync` | When `true`, syncs metadata immediately after creation. | `true` |

## Connection Fields

| Property | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `name` | string | Yes |  | Datastore label shown in the UI. |
| `teams` | array of strings | No | `Public` | Teams that can access the datastore. Left out, the datastore is assigned to the **Public** team. |
| `root_path` | string | Yes | `/` | Subfolder inside the container where the data lives (source) or where Qualytics writes its outputs (enrichment). |
| `enrichment_only` | boolean | Yes |  | `false` for a source datastore. `true` for an enrichment datastore. |
| `connection.name` | string | Yes |  | Saved-connection label. |
| `connection.type` | string | Yes |  | Connector identifier. Always `abfs` for Azure Data Lake Storage Gen2. |
| `connection.uri` | string | Yes |  | Container-level ABFS URI in the format `abfss://<container>@<account>.dfs.core.windows.net`. |
| `connection.access_key` | string | Yes |  | Account Name when `authentication_type` is `SHARED_KEY`. Client ID when `authentication_type` is `SERVICE_PRINCIPAL`. |
| `connection.secret_key` | string | Yes |  | Access Key when `authentication_type` is `SHARED_KEY`. Client Secret when `authentication_type` is `SERVICE_PRINCIPAL`. |
| `connection.parameters.authentication_type` | string | No | `SHARED_KEY` | `SHARED_KEY` (Shared Key) or `SERVICE_PRINCIPAL`. |
| `connection.parameters.tenant_id` | string | If `SERVICE_PRINCIPAL` |  | Directory (Tenant) ID of the Microsoft Entra ID app registration. |

> **Note: About trigger_sync**
>
> The `trigger_sync: true` flag in the examples below tells Qualytics to immediately sync metadata after creating the datastore (equivalent to ticking the **Initiate Sync** toggle in the UI). Set it to `false` if you want to defer the first sync.

---

## Create a Source Datastore

A source datastore uses a read-only identity. Set `enrichment_only: false` to mark the datastore as a source.

**Tab: Shared Key (SHARED_KEY)**

The default authentication type. Use when you can provide the storage account name and its access key.

> **Example: Example request and response**
>
> **Request**:
>
> ```bash
> curl -X POST "https://your-instance.qualytics.io/api/datastores" \
>   -H "Authorization: Bearer YOUR_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "name": "Orders Lake",
>     "teams": ["Engineering"],
>     "root_path": "/",
>     "enrichment_only": false,
>     "trigger_sync": true,
>     "connection": {
>       "name": "acme_adls_lake",
>       "type": "abfs",
>       "uri": "abfss://raw@acmelake.dfs.core.windows.net",
>       "access_key": "acmelake",
>       "secret_key": "<storage-account-access-key>",
>       "parameters": {
>         "authentication_type": "SHARED_KEY"
>       }
>     }
>   }'
> ```
>
> **Response** (200 OK):
>
> ```json
> {
>   "id": 42,
>   "name": "Orders Lake",
>   "type": "abfs",
>   "store_type": "dfs",
>   "root_path": "/",
>   "enrichment_only": false,
>   "connected": true
> }
> ```

**Tab: Service Principal (SERVICE_PRINCIPAL)**

Authenticate via a Microsoft Entra ID app registration. Set `parameters.authentication_type` to `"SERVICE_PRINCIPAL"` and provide the Client ID, Client Secret, and Tenant ID.

> **Example: Example request and response**
>
> **Request**:
>
> ```bash
> curl -X POST "https://your-instance.qualytics.io/api/datastores" \
>   -H "Authorization: Bearer YOUR_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "name": "Orders Lake",
>     "teams": ["Engineering"],
>     "root_path": "/",
>     "enrichment_only": false,
>     "trigger_sync": true,
>     "connection": {
>       "name": "acme_adls_lake",
>       "type": "abfs",
>       "uri": "abfss://raw@acmelake.dfs.core.windows.net",
>       "access_key": "00000000-0000-0000-0000-000000000000",
>       "secret_key": "<client-secret>",
>       "parameters": {
>         "authentication_type": "SERVICE_PRINCIPAL",
>         "tenant_id": "11111111-1111-1111-1111-111111111111"
>       }
>     }
>   }'
> ```
>
> **Response** (200 OK):
>
> ```json
> {
>   "id": 42,
>   "name": "Orders Lake",
>   "type": "abfs",
>   "store_type": "dfs",
>   "root_path": "/",
>   "enrichment_only": false,
>   "connected": true
> }
> ```

**Tab: Existing Connection**

Reuse a previously created Azure Data Lake Storage connection by passing its `connection_id`. The authentication type and credentials are reused from the existing connection. You only specify datastore-level fields like `name`, `teams`, and `root_path`.

> **Example: Example request and response**
>
> **Request**:
>
> ```bash
> curl -X POST "https://your-instance.qualytics.io/api/datastores" \
>   -H "Authorization: Bearer YOUR_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "name": "Finance Lake",
>     "teams": ["Finance Team"],
>     "root_path": "/finance/",
>     "enrichment_only": false,
>     "trigger_sync": true,
>     "connection_id": 7
>   }'
> ```
>
> **Response** (200 OK):
>
> ```json
> {
>   "id": 43,
>   "name": "Finance Lake",
>   "type": "abfs",
>   "store_type": "dfs",
>   "root_path": "/finance/",
>   "enrichment_only": false,
>   "connected": true
> }
> ```

---

## Create an Enrichment Datastore

An enrichment datastore uses a read-write identity. Set `enrichment_only: true` to create the datastore for enrichment.

**Tab: Shared Key (SHARED_KEY)**

Provide the storage account name and its access key. The account must have write permissions on the container where Qualytics writes enrichment files.

> **Example: Example request and response**
>
> **Request**:
>
> ```bash
> curl -X POST "https://your-instance.qualytics.io/api/datastores" \
>   -H "Authorization: Bearer YOUR_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "name": "Lake Enrichment",
>     "teams": ["Engineering"],
>     "root_path": "/qualytics-enrichment/",
>     "enrichment_only": true,
>     "trigger_sync": true,
>     "connection": {
>       "name": "acme_adls_enrichment",
>       "type": "abfs",
>       "uri": "abfss://raw@acmelake.dfs.core.windows.net",
>       "access_key": "acmelake",
>       "secret_key": "<storage-account-access-key>",
>       "parameters": {
>         "authentication_type": "SHARED_KEY"
>       }
>     }
>   }'
> ```
>
> **Response** (200 OK):
>
> ```json
> {
>   "id": 44,
>   "name": "Lake Enrichment",
>   "type": "abfs",
>   "store_type": "dfs",
>   "root_path": "/qualytics-enrichment/",
>   "enrichment_only": true,
>   "connected": true
> }
> ```

**Tab: Service Principal (SERVICE_PRINCIPAL)**

Use a Microsoft Entra ID app registration whose role assignment grants read and write access on the container (for example, **Storage Blob Data Contributor**). Set `parameters.authentication_type` to `"SERVICE_PRINCIPAL"` and provide the Client ID, Client Secret, and Tenant ID.

> **Example: Example request and response**
>
> **Request**:
>
> ```bash
> curl -X POST "https://your-instance.qualytics.io/api/datastores" \
>   -H "Authorization: Bearer YOUR_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "name": "Lake Enrichment",
>     "teams": ["Engineering"],
>     "root_path": "/qualytics-enrichment/",
>     "enrichment_only": true,
>     "trigger_sync": true,
>     "connection": {
>       "name": "acme_adls_enrichment",
>       "type": "abfs",
>       "uri": "abfss://raw@acmelake.dfs.core.windows.net",
>       "access_key": "00000000-0000-0000-0000-000000000000",
>       "secret_key": "<client-secret>",
>       "parameters": {
>         "authentication_type": "SERVICE_PRINCIPAL",
>         "tenant_id": "11111111-1111-1111-1111-111111111111"
>       }
>     }
>   }'
> ```
>
> **Response** (200 OK):
>
> ```json
> {
>   "id": 44,
>   "name": "Lake Enrichment",
>   "type": "abfs",
>   "store_type": "dfs",
>   "root_path": "/qualytics-enrichment/",
>   "enrichment_only": true,
>   "connected": true
> }
> ```

**Tab: Existing Connection**

Reuse a previously created Azure Data Lake Storage connection by passing its `connection_id`. The authentication type and credentials are reused from the existing connection. You only specify datastore-level fields like `name`, `teams`, and `root_path`.

> **Example: Example request and response**
>
> **Request**:
>
> ```bash
> curl -X POST "https://your-instance.qualytics.io/api/datastores" \
>   -H "Authorization: Bearer YOUR_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "name": "Finance Enrichment",
>     "teams": ["Finance Team"],
>     "root_path": "/qualytics-enrichment/",
>     "enrichment_only": true,
>     "trigger_sync": true,
>     "connection_id": 9
>   }'
> ```
>
> **Response** (200 OK):
>
> ```json
> {
>   "id": 45,
>   "name": "Finance Enrichment",
>   "type": "abfs",
>   "store_type": "dfs",
>   "root_path": "/qualytics-enrichment/",
>   "enrichment_only": true,
>   "connected": true
> }
> ```

---

## Other Operations

For get, list, update, delete, test connection, and link-enrichment endpoints, see the [Datastore API](https://userguide.qualytics.io/source-datastore/datastore/api/) reference.

*Last updated: November 3, 2026.*
