DQP Prerequisites

Alation Cloud Service Applies to Alation Cloud Service instances of Alation

Customer Managed Applies to customer-managed instances of Alation

Contact the Forward Deployed Engineering team or your Alation Account Team to obtain the Data Quality Processor (DQP) installer. You will be provided with a download link. The installer must then be uploaded to your Alation Services Manager instance.

Alation Service Manager (ASM)

ASM must be installed and configured first. DQP is then installed and configured on ASM. Contact the Forward Deployed Engineering team or your Alation Account Team to obtain ASM.

Network Requirements

ASM requires inbound/outbound connections on port 443 to be open for DQP to function. This is because DQP leverages APIs that create HTTPS inbound/outbound traffic between ASM and your Alation instance.

Cataloged Data Sources

The source system and the corresponding data associated with the DQ rules you will create must be set up as a cataloged Alation data source, and MDE must have been run at least once on that source system.

Data Quality Processor for Snowflake

To begin using Alation DQP with Snowflake, data metric functions (DMFs) are required. Please refer to Snowflake DMFs documentation for more information. Once DQ rules are created in Snowflake and executed, Alation’s DQP ingests results, maps them to the corresponding catalog assets, applies RAG thresholds, and overlays the insights with metadata.

Note

For Snowflake users, Snowflake Enterprise Edition is required to use Data Metric Functions (DMFs). DMFs are a mandatory requirement for DQP.

Data Quality Processor for Databricks

Lakehouse Monitoring is required to begin using Alation DQP with Databricks. This needs your workspace to be enabled for Unity Catalog, and you must have access to Databricks SQL. Please refer to Databricks Lakehouse Monitoring documentation for details. Once DQ rules are created in Databricks and executed, Alation’s DQP ingests results, maps them to the corresponding catalog assets, applies RAG (Red/Amber/Green) thresholds, and overlays the insights with metadata.

Compose

You will need to have Alation Compose configured with your data source. Compose is used by DQP to automatically execute the SQL needed to retrieve and publish the data quality results. For example, if Snowflake is the source of your data quality information, you will need to have Compose enabled and configured with the correct connection details and credentials for your Snowflake instance.