Skip to content

Datastore Settings

Qualytics allows you to manage your datastore efficiently by editing source datastore information, linking an enrichment datastore for enhanced insights, establishing new connections to expand data sources, choosing connectors to integrate diverse data, adjusting the quality score to ensure data accuracy, and deleting the store. This ensures flexibility and control over your data management processes within the platform.

Let's get started 🚀

Step 1: Select a source datastore from the side menu for which you would like to manage the settings.

add-datastore add-datastore

Step 2: Click on the Settings icon from the top right window. A drop-down menu will appear with the following options:

  1. Edit
  2. Enrichment
  3. Score
  4. Delete

settings settings

Edit Datastore

The Edit Datastore setting allows users to modify the connection details of the datastore. This includes updating the host, port, SID, username, password, schema, and any associated teams.

Note

Connection details can vary based on the type of datastore being edited. For example, details for BigQuery will differ from Snowflake or Athena.

Step 1: Click on the Edit option

edit-datastore edit-datastore

Step 2: After selecting the Edit option, a modal window will appear, displaying the connection details. This window allows you to modify any specific connection details.

connection-details connection-details

Step 3: After editing the connection details, click on the Save button.

save-datastore save-datastore

An enrichment datastore is a database used to enhance your existing data by adding additional, relevant information. This helps you to provide more comprehensive insight into data and improve data accuracy.

You have the option to link an enrichment datastore to your existing source datastore. However, some datastores cannot be linked as enrichment datastores. For example, Oracle, Athena, and Timescale cannot be used for this purpose.

Step 1: Click on the Enrichment from the dropdown list.

enrichment enrichment

Step 2: A modal window- Link Enrichment Datastore will appear, providing you with two options to link an enrichment datastore.

REF. FIELDS ACTIONS
1. Prefix Add a prefix name to uniquely identify tables/files when Qualytics writes metadata from the source datastore to your enrichment datastore.
2. Toggle Button for existing enrichment datastore Toggle ON to link the source datastore to an existing enrichment datastore, or toggle OFF to link it to a brand new enrichment datastore.
3. Name Give a name for the enrichment datastore.
4. Toggle Button for using an existing connection Toggle ON to reuse credentials from an existing connection, or toggle OFF to create a new enrichment from scratch.
5. Connector Select a datastore connector as Teradata from the dropdown list.

link-enrichment link-enrichment

If the toggle for Use an existing connection is turned off, then this will prompt you to link a new enrichment datastore from scratch without using existing connection details.

Step 1: Select the Enrichment datastore from the connector dropdown list and add connection details such as URI, access key, secret key, root path, and teams.

Note

Connection details can vary from datastore to datastore. For illustration, we have demonstrated linking Amazon S3 as a new enrichment datastore.

select-new-enrichment select-new-enrichment

Step 2: The configuration form will expand, requesting credential details before establishing the connection.

connector-creds connector-creds

Step 3: After adding the source datastore details, click on the Test Connection button to check and verify its connection.

test-connection test-connection

If the credentials and provided details are verified, a success message will be displayed indicating that the connection has been verified.

Step 4: Click on the Save button.

save-enrichment save-enrichment

Step 5: After clicking on the Save button a modal window will appear Your Datastore has been successfully updated

success success

If the toggle for Use an existing enrichment datastore is turned on, you will be prompted to link the enrichment datastore using existing connection details.

Step 1: Select an existing enrichment datastore from the dropdown list.

existing-connection existing-connection

Step 2: View and check the connection details of the enrichment and click on the Save button.

save save

Step 3: After clicking on the Save button a modal window will appear Your Datastore has been successfully updated

success success

Quality Score Settings

Quality Scores are quantified measures of data quality calculated at the field and container levels recorded as time series to enable tracking of changes over time. Scores range from 0-100 with higher values indicating superior quality. These scores integrate eight distinct factors providing a granular analysis of the attributes that impact the overall data quality.

Each field receives a total quality score based on eight key factors each evaluated on a 0-100 scale. The overall score is a composite reflecting the relative importance and configured weights of these factors:

  • Completeness: Measures the average completeness of a field across all profiles.

  • Coverage: Assesses the adequacy of data quality checks for the field.

  • Conformity: Checks alignment with standards defined by quality checks.

  • Consistency: Ensures uniformity in type and scale across all data representations.

  • Precision: Evaluates the resolution of field values against defined quality checks.

  • Timeliness: Gauges data availability according to schedule inheriting the container's timeliness.

  • Volumetrics: Analyzes consistency in data size and shape over time inheriting the container's volumetrics.

  • Accuracy: Determines the fidelity of field values to their real-world counterparts.

The Quality Score Settings allow users to tailor the impact of each quality factor on the total score by adjusting their weights allowing the scoring system to align with your organization’s data governance priorities.

Step 1: Click on the Score option in the settings icon.

score score

Step 2: A modal window "Quality Score Settings" will appear.

score-settings score-settings

Step 3: The Decay Period slider sets the time frame over which the system evaluates historical data to determine the quality score. The decay period for considering past data events defaults to 180 days but can be customized to fit your operational needs ensuring the scores reflect the most relevant data quality insights.

decay-period decay-period

Step 4: Adjust the Factor Weights using the sliding bar. The factor weights determine the importance of different data quality aspects.

factor-weights factor-weights

Step 5: Click on the Save button to save the quality score settings.

score-save score-save

Delete Datastore

The Delete Datastore action permanently removes a datastore and all associated profiles, checks, and anomalies. This action cannot be undone and requires confirmation by typing the datastore name before proceeding.

Step 1: Click on the Delete option in the settings icon.

delete delete

Step 2: A modal window Delete Datastore will appear.

delete-datastore delete-datastore

Step 3: Enter the Name of the datastore in the given field (confirmation check) and then click on the I’M SURE, DELETE THIS DATASTORE button to delete the datastore.

confirm-delete confirm-delete