Datastore Settings
Qualytics allows you to manage your datastore efficiently by editing source datastore information, linking an enrichment datastore for enhanced insights, establishing new connections to expand data sources, choosing connectors to integrate diverse data, adjusting the quality score to ensure data accuracy, and deleting the store. This ensures flexibility and control over your data management processes within the platform.
Let's get started 🚀
Navigation to Settings
Step 1: Select a source datastore from the side menu for which you would like to manage the settings.
Step 2: Click on the Settings icon from the top right window. A drop-down menu will appear with the following options:
- Edit
- Enrichment
- Score
- Delete
Edit Datastore
The Edit Datastore setting allows users to modify the connection details of the datastore. This includes updating the host, port, SID, username, password, schema, and any associated teams.
Note
Connection details can vary based on the type of datastore being edited. For example, details for BigQuery will differ from Snowflake or Athena.
Step 1: Click on the Edit option
Step 2: After selecting the Edit option, a modal window will appear, displaying the connection details. This window allows you to modify any specific connection details.
Step 3: After editing the connection details, click on the Save button.
Link Enrichment Datastore
An enrichment datastore is a database used to enhance your existing data by adding additional, relevant information. This helps you to provide more comprehensive insight into data and improve data accuracy.
You have the option to link an enrichment datastore to your existing source datastore. However, some datastores cannot be linked as enrichment datastores. For example, Oracle, Athena, and Timescale cannot be used for this purpose.
Step 1: Click on the Enrichment from the dropdown list.
Step 2: A modal window- Link Enrichment Datastore will appear, providing you with two options to link an enrichment datastore.
| REF. | FIELDS | ACTIONS |
|---|---|---|
| 1. | Prefix | Add a prefix name to uniquely identify tables/files when Qualytics writes metadata from the source datastore to your enrichment datastore. |
| 2. | Toggle Button for existing enrichment datastore | Toggle ON to link the source datastore to an existing enrichment datastore, or toggle OFF to link it to a brand new enrichment datastore. |
| 3. | Name | Give a name for the enrichment datastore. |
| 4. | Toggle Button for using an existing connection | Toggle ON to reuse credentials from an existing connection, or toggle OFF to create a new enrichment from scratch. |
| 5. | Connector | Select a datastore connector as Teradata from the dropdown list. |
Option I: Link New Enrichment
If the toggle for Use an existing connection is turned off, then this will prompt you to link a new enrichment datastore from scratch without using existing connection details.
Step 1: Select the Enrichment datastore from the connector dropdown list and add connection details such as URI, access key, secret key, root path, and teams.
Note
Connection details can vary from datastore to datastore. For illustration, we have demonstrated linking Amazon S3 as a new enrichment datastore.
Step 2: The configuration form will expand, requesting credential details before establishing the connection.
Step 3: After adding the source datastore details, click on the Test Connection button to check and verify its connection.
If the credentials and provided details are verified, a success message will be displayed indicating that the connection has been verified.
Step 4: Click on the Save button.
Step 5: After clicking on the Save button a modal window will appear Your Datastore has been successfully updated
Option II: Link Existing Connection
If the toggle for Use an existing enrichment datastore is turned on, you will be prompted to link the enrichment datastore using existing connection details.
Step 1: Select an existing enrichment datastore from the dropdown list.
Step 2: View and check the connection details of the enrichment and click on the Save button.
Step 3: After clicking on the Save button a modal window will appear Your Datastore has been successfully updated
Quality Score Settings
Quality Scores are quantified measures of data quality calculated at the field and container levels recorded as time series to enable tracking of changes over time. Scores range from 0-100 with higher values indicating superior quality. These scores integrate eight distinct factors providing a granular analysis of the attributes that impact the overall data quality.
Each field receives a total quality score based on eight key factors each evaluated on a 0-100 scale. The overall score is a composite reflecting the relative importance and configured weights of these factors:
-
Completeness: Measures the average completeness of a field across all profiles.
-
Coverage: Assesses the adequacy of data quality checks for the field.
-
Conformity: Checks alignment with standards defined by quality checks.
-
Consistency: Ensures uniformity in type and scale across all data representations.
-
Precision: Evaluates the resolution of field values against defined quality checks.
-
Timeliness: Gauges data availability according to schedule inheriting the container's timeliness.
-
Volumetrics: Analyzes consistency in data size and shape over time inheriting the container's volumetrics.
-
Accuracy: Determines the fidelity of field values to their real-world counterparts.
The Quality Score Settings allow users to tailor the impact of each quality factor on the total score by adjusting their weights allowing the scoring system to align with your organization’s data governance priorities.
Step 1: Click on the Score option in the settings icon.
Step 2: A modal window "Quality Score Settings" will appear.
Step 3: The Decay Period slider sets the time frame over which the system evaluates historical data to determine the quality score. The decay period for considering past data events defaults to 180 days but can be customized to fit your operational needs ensuring the scores reflect the most relevant data quality insights.
Step 4: Adjust the Factor Weights using the sliding bar. The factor weights determine the importance of different data quality aspects.
Step 5: Click on the Save button to save the quality score settings.
Delete Datastore
The Delete Datastore action permanently removes a datastore and all associated profiles, checks, and anomalies. This action cannot be undone and requires confirmation by typing the datastore name before proceeding.
Step 1: Click on the Delete option in the settings icon.
Step 2: A modal window Delete Datastore will appear.
Step 3: Enter the Name of the datastore in the given field (confirmation check) and then click on the I’M SURE, DELETE THIS DATASTORE button to delete the datastore.
