Configure Lineage¶
Alation Cloud Service Applies to Alation Cloud Service instances of Alation
Customer Managed Applies to customer-managed instances of Alation
To configure lineage for dbt Cloud and dbt Core in Alation, follow these steps:
Fetch dbt Projects/Environments and Map Datasources¶
To fetch dbt projects/environments, perform these steps:
Open Lineage Settings in your dbt Gen2 ELT source.
Go to the Lineage Configuration tab
Click Run to fetch all Alation data sources that match with projects or environments within projects present in dbt.
The retrieved list of projects/environments appears in the Lineage extracted sources table.
Run Lineage Extraction¶
Lineage extraction is based on your project selection during the metadata extraction configuration.
For models, Alation displays lineage of all related tables. By default, the first table in the list is selected, but you can change it using the dropdown menu.
Lineage extraction fetches lineage metadata (additional metadata such as, Jinja code and SQL query) from dbt and generates lineage between various sources and targets such as RDBMS tables, views, and columns.
To run lineage extraction, perform these steps:
Open Lineage Settings in your dbt Gen2 ELT source.
Go to the Lineage Configuration tab.
Click Run extraction.
You can schedule extraction depending on your requirements. Scheduling lineage extraction is similar to scheduling metadata extraction. For details, see the Schedule Extraction section of Metadata Extraction.
View the Lineage Job History¶
You can view the status of the lineage extraction jobs after you run the extraction or after Alation triggers the lineage extraction as per the schedule.
To view the status of extraction, go to Lineage Extraction > Lineage Job History on the Settings page of your dbt Gen2 ELT source. The Extraction job status table displays the status of project extraction and corresponding lineage extraction.
Click the View Details link to view a detailed report of lineage extraction. If there are errors, the Job errors table displays the error category, error message, and a hint (ways to resolve the issue). Follow the instructions under the Hints column to resolve the error.
In some cases, Generate Error Report link is displayed above the Job errors table. Click the Generate Error Report link above the Job errors table to generate an archive (.zip) containing CSV files for different error categories, such as Data and Connection errors. Click Download Error Report to download the files.
Understand Lineage Extraction from dbt¶
To view the lineage for the tables associated with a model, you can select the table from the Lineage tab on the model catalog page.
By default, the catalog page displays the lineage for the first table listed in the Source System information section.
Note
Objects that are related to the model but aren’t cataloged yet are listed without links in the Source System information section.
To learn more about viewing lineage in the Alation catalog, see Discover Lineage.
For information on how to configure Lineage using Alation user interface, see Configure Lineage.
View Data Health Information from dbt¶
The connector pushes the MDE model execution and test run results to associated data source tables. To view the health information for any table with associated Data Health rule, navigate to the source table using the link on the Source System information section on the model catalog page and click Data Health.
Any failure in model execution gets propagated to downstream objects in the lineage graph making it easy to find the root cause of the failure.
Here’s an example:
To understand more about viewing data health within the Alation data catalog, see View Data Health.