# Amazon S3 Add an Amazon S3 Source Datastore

Source: https://userguide.qualytics.io/source-datastore/datastore/dfs/amazon-s3/how-tos/add-source/

A source datastore is a storage location Qualytics connects to so it can profile, scan, and monitor data. Adding S3 as a source lets Qualytics read files directly from your bucket and run quality operations on the data they contain.

Before you start, review the [Amazon S3 Permissions](https://userguide.qualytics.io/source-datastore/datastore/dfs/amazon-s3/permissions/) and the available [Authentication](https://userguide.qualytics.io/source-datastore/datastore/dfs/amazon-s3/authentication/) methods.

> **Tip**
>
> An S3 datastore can also be used as an enrichment datastore. When linked to a source, the S3 datastore becomes that source's enrichment destination for anomalies and metadata. Link one during creation via [Link Enrichment on Datastore Creation](https://userguide.qualytics.io/source-datastore/enrichment-datastore/how-tos/link-during-creation/), or afterwards via [Link an Enrichment Destination](https://userguide.qualytics.io/source-datastore/enrichment-datastore/how-tos/link-enrichment/).

## Field reference

The **Add Datastore** page shows the sections below when Amazon S3 is selected. When reusing an existing connection, the **Connection Properties**, **Authentication**, and **Secrets Management** sections come already filled in and read-only: Qualytics has already validated those credentials, so you fill in only the **Location** and the **General** fields. To change a saved connection's credentials, edit the connection through the [Manage Connections](https://userguide.qualytics.io/settings/connections/manage-connections/) page; edits there apply to every datastore that reuses the connection.

### Connection Properties

These fields define where the bucket lives. They belong to the connection: when reusing an existing connection, they come already filled in and read-only.

| Field | Required | Type | Description |
| --- | --- | --- | --- |
| **Connection Name** | Yes | Text | A label for the saved connection (e.g., `acme_s3_lake`), so other datastores can reuse it later. |
| **URI** | Yes | Text | The bucket-level URI, in the form `s3://<bucket_name>`. Leave out any folder path here and use **Root Path** below to scope to a subfolder. The region is worked out from the bucket. |

### Authentication

Choose how Qualytics authenticates to AWS. Setting **Type** changes the credential fields shown below it, so pick the tab that matches your choice. These fields also belong to the connection: already filled in and read-only when reusing one. See [Authentication](https://userguide.qualytics.io/source-datastore/datastore/dfs/amazon-s3/authentication/) for how each method works.

**Tab: Access Key**

| Field | Required | Type | Description |
| --- | --- | --- | --- |
| **Type** | No | Option | Set to **Access Key**, which is the default (`SHARED_KEY` in the API). |
| **Access Key** | Yes | Text | The access key ID of an IAM user holding the permissions listed in [Permissions](https://userguide.qualytics.io/source-datastore/datastore/dfs/amazon-s3/permissions/). |
| **Secret Key** | Yes | Text | The secret access key that goes with it. |

**Tab: IAM Role**

| Field | Required | Type | Description |
| --- | --- | --- | --- |
| **Type** | No | Option | Set to **IAM Role** (`IAM_ROLE` in the API). Qualytics assumes the role and uses short-lived credentials that refresh on their own. |
| **Role ARN** | Yes | Text | The ARN of the IAM role to assume (e.g., `arn:aws:iam::123456789012:role/MyRole`). |
| **External ID** | No | Text | Fill this in only when the role's trust policy requires an external ID. |

> **Note: AWS-only**
>
> **IAM Role** appears in **Type** only on AWS and local deployments. On Azure and GCP deployments, only **Access Key** is offered.

### Secrets Management

This group is optional: use it only if you want Qualytics to pull credentials from a secrets manager instead of typing them into the form. Turn on **HashiCorp Vault** to show the fields below. Despite the label, any secrets manager that exposes a compatible REST API works, not only HashiCorp Vault; see [HashiCorp Vault](https://userguide.qualytics.io/source-datastore/datastore/connection/hashicorp-vault/deep-dive/introduction/). It also belongs to the connection: read-only when reusing an existing connection.

| Field | Required | Type | Description |
| --- | --- | --- | --- |
| **Login URL** | Yes | Text | The full URL of the login endpoint (e.g., `https://vault.example.com/v1/auth/approle/login`). With Vault namespaces, add the namespace after `/v1/`, as in `/v1/<namespace>/auth/approle/login`. |
| **Credentials Payload** | Yes | Text | A JSON body containing the credentials Vault expects (e.g., `{"role_id":"...","secret_id":"..."}`). |
| **Token JSONPath** | Yes | Text | The JSONPath that extracts the client token from Vault's response. Defaults to `$.auth.client_token`. |
| **Secret URL** | Yes | Text | The full URL of the secret, including `https://` and the host (e.g., `https://vault.example.com/v1/secret/data/s3`). With Vault namespaces, add the namespace after `/v1/`. |
| **Token Header Name** | Yes | Text | The HTTP header name used to send the token. Defaults to `X-Vault-Token`. |
| **Data JSONPath** | Yes | Text | The JSONPath that extracts the key/value pairs from Vault's response. Use `$.data.data` for a KV version 2 secret (its API path contains `/data/`) and `$.data` for KV version 1 or the database secrets engine. The form pre-fills `$.data`. |

> **Note**
>
> Once the secrets manager is configured, reference any secret from a **Connection Properties** or **Authentication** field with `${key}`, for example `${secret_key}`. Qualytics resolves it each time the connection is opened, so a changed value takes effect on the next connection. Write the reference with braces (`${key}`, never `$.key`) and match the key name's case. Setup steps, the namespace rule, and a troubleshooting page are on the [HashiCorp Vault](https://userguide.qualytics.io/source-datastore/datastore/connection/hashicorp-vault/deep-dive/introduction/) pages.

### Location

Pick the folder inside the bucket Qualytics should read from. You fill this in on both flows.

| Field | Required | Type | Description |
| --- | --- | --- | --- |
| **Root Path** | Yes | Option | The folder inside the bucket where the data lives (e.g., `/raw/orders/`). Defaults to `/`, which reads from the bucket root. Click the refresh icon to load the folders the credentials can see. |

### General

Common fields for every source datastore, shown below the **Location** section. You fill these in on both flows.

| Field | Required | Type | Description |
| --- | --- | --- | --- |
| **Name Template** | No | Text | Defines the naming pattern for the source datastore being created. Left empty, the datastore is named from the connection name and the root path. |
| **Group** | No | Option | Organizes your datastores under a shared group in the navigation tree. Select an existing group or create a new one with the **Add New Group** toggle. |
| **Teams** | Yes | Option | Select one or more teams to associate with this source datastore. |
| **Initiate Sync** | No | Checkbox | Automatically sync the datastore to detect containers and fields after creation. |

Below the form, an information banner lists the **Public addresses** and **Private addresses** your datastore connections originate from. Allow the ones that match your network setup through your security groups or firewall rules; see [How Connections Work](https://userguide.qualytics.io/source-datastore/datastore/connection/how-it-works/#finding-the-addresses-to-allow) for where the addresses come from.

## Steps

There are two ways to set up the connection: reuse a connection you already saved (**Existing Connection**) or create a new one from scratch (**New Connection**). The tabs below walk through each option; pick the one you want to follow. Each field is described in the [Field reference](https://userguide.qualytics.io/source-datastore/datastore/dfs/amazon-s3/how-tos/add-source/#field-reference) above.

**Tab: New Connection**

**Step 1**: Navigate to the **Datastores** page.

**Step 2**: Click the **Add**  button at the top-right corner and choose **Source** .

**Step 3**: The **Add Datastore** page opens.

**Step 4**: Select **New Connection** next to the **Search**  field.

**Step 5**: Select **Amazon S3** from the connector grid. Use the search field to filter connectors by name.

**Step 6**: Fill in the **Connection Properties**: the **Connection Name** and the **URI**.

**Step 7**: Fill in the **Authentication** fields for the **Type** you choose.

**Step 8**: Optionally, expand **Secrets Management** to retrieve credentials from a secrets manager.

**Step 9**: Fill in the **Location** field (**Root Path**) and the **General** fields.

**Step 10**: Click **Test connection**. A success message confirms that the connection has been verified.

> **Info**
>
> The **Finish** and **Next** buttons stay disabled until the connection test passes on the current values. If the test fails, see [Troubleshooting](https://userguide.qualytics.io/source-datastore/datastore/dfs/amazon-s3/troubleshooting/).

**Step 11**: Click **Finish** to create the datastore.

> **Tip**
>
> To link an enrichment destination so Qualytics can store anomalies and metadata from the first operation, click **Next** instead of **Finish**. See [Link Enrichment on Datastore Creation](https://userguide.qualytics.io/source-datastore/enrichment-datastore/how-tos/link-during-creation/).

**Step 12**: A success dialog confirms that your datastore has been added. Click **Go to your datastore** to open its page.

**Tab: Existing Connection**

**Step 1**: Navigate to the **Datastores** page.

**Step 2**: Click the **Add**  button at the top-right corner and choose **Source** .

**Step 3**: The **Add Datastore** page opens.

**Step 4**: Select **Existing Connection** next to the **Search**  field.

**Step 5**: Select the saved S3 connection from the grid. Use the search field to filter connections by name. The **Connection Properties**, **Authentication**, and **Secrets Management** sections come already filled in and read-only.

> **Tip: Start a new connection from this one**
>
> To use the selected connection as a starting point for a brand-new connection instead, click the **Duplicate as a new connection**  button on the selected connection. The form switches to **New Connection** mode with the connection's settings already filled in for you to adjust.

**Step 6**: Fill in the **Location** field (**Root Path**) and the **General** fields. These are the only fields left to fill in.

**Step 7**: Click **Test connection**. A success message confirms that the connection has been verified.

> **Info**
>
> The **Finish** and **Next** buttons stay disabled until the connection test passes on the current values. If the test fails, see [Troubleshooting](https://userguide.qualytics.io/source-datastore/datastore/dfs/amazon-s3/troubleshooting/).

**Step 8**: Click **Finish** to create the datastore.

> **Tip**
>
> To link an enrichment destination so Qualytics can store anomalies and metadata from the first operation, click **Next** instead of **Finish**. See [Link Enrichment on Datastore Creation](https://userguide.qualytics.io/source-datastore/enrichment-datastore/how-tos/link-during-creation/).

**Step 9**: A success dialog confirms that your datastore has been added. Click **Go to your datastore** to open its page.

*Last updated: November 3, 2026.*
