Skip to content

Materialize Operation

Materialize Operation captures snapshots of selected containers from a source datastore and exports them to an enrichment datastore for seamless data loading. Users can run it instantly or schedule it at set intervals, ensuring structured data is readily available for analysis and integration.

Let’s get started 🚀

Step 1: Select a source datastore from the side menu to capture and export containers for the Materialize Operation.

select select

For demonstration purposes, we have selected the “COVID-19 Data” Snowflake source datastores.

datastore datastore

Step 2: After selecting a datastore, a bottom-up menu appears on the right side of the interface. Click Enrichment Operations next to the Enrichment Datastore and select Materialize.

material material

Step 3: After clicking Materialize a modal window appears, allowing you to configure the data export settings for the Materialize Operation.

window window

Step 4: Select tables to materialize all tables, specific tables, or tables by tag, then click Next.

next next

Step 5: Configure Record Limit set the maximum number of records to be materialized per table.

record record

Run Now

Click Run Now to instantly materialize selected containers.

run run

Schedule

Step 1: Click Schedule to configure scheduling options for the Materialize Operation.

schedule schedule

Step 2: Configure the scheduling preferences for the Materialize Operation.

  • Hourly: Runs every set number of hours at a specified minute. (e.g., Every 1 hour at 00 minutes).

  • Daily: Runs once per day at a specific UTC time. (e.g., Every day at 00:00 UTC).

  • Weekly: Runs on selected weekdays at a set time. (e.g., Every Sunday and Friday at 00:00 UTC).

  • Monthly: Runs on a specific day of the month at a set time. (e.g., 1st day of every month at 00:00 UTC).

  • Advanced: Use Cron expressions for custom scheduling. (e.g., 0 12 * * 1-5 runs at 12 PM, Monday to Friday).

time time

Step 3: Define the Schedule Name to identify the scheduled Materialize Operation when it runs.

name name

Step 4: Click Schedule to finalize and schedule the Materialize Operation.

schedule2 schedule2

Review Materialized Data

Step 1: Once the selected containers are materialized, go to Enrichment Datastores from the left menu.

preview preview

Step 2: In the Enrichment Datastores section, select the datastore where you materialized the snapshot. The materialized containers will now be visible.

data data

Step 3: Click on the materialized files to review the snapshot. For demonstration, we have selected the "materialized_field_profiles" file.

The materialized data is displayed in a table format, showing key details about the selected containers. It typically includes columns indicating data structure, completeness, and uniqueness. You can use this data for analysis, validation, and integration.

preview2 preview2