How Bots Work

Alation Cloud Service Applies to Alation Cloud Service instances of Alation

Customer Managed Applies to customer-managed instances of Alation

A Bot is an automation that monitors the catalog content and triggers notifications, assigns tasks or takes other actions.

The architecture is illustrated below. On the left is Alation Services Manager (ASM), the engine which provides the foundational components for services like Bots including UI, scheduling, and containerization. On the right are the Alation components that Bots interact with: The main application, and Alation Analytics (cloud and on-premises).

../../../_images/FDEbotsArch.png

Bots have three main components:

  1. The Bot framework - Bots are developed using Python and use a Docker containerized environment provided by the Alation Services Manager (ASM).

  2. Bot Configuration - A Bot’s configuration contains the rules and actions that the Bot will follow. Bots are created, configured and managed in the Bot Manager in ASM.

  3. Bot Policy - Each Bot requires its own Policy be created in Alation. Bot Policies allows catalog users to clearly see Bot descriptions and configurations.

The catalog content that a Bot monitors is configurable using simple rules, expressed as SQL. Each Bot has one policy and a Bot can contain multiple rules.

Here is an example Bot policy. This Bot finds the 25 most popular table object pages that are missing a Title and Description. A list of tables found to be in violation (if any) can be viewed using the link provided:

../../../_images/FDEbotsPolicy.png

To read content from the catalog and take action, Bots work in conjunction with Alation Analytics (AAv2) and Alation APIs respectively. A Bot’s activity is defined using a Bot Rule which achieves three things:

  1. AAv2 is queried to monitor and check catalog content. This is achieved by running SQL select statements against AAv2 to obtain the required information. Example: For a data source ownership Bot, the query would select a list of all data sources and associated attributes.

  2. The result of the query is then evaluated against the conditions in the Bot Rules. Rules can be complex - up to 4 Bot rules per configuration can be used. The rules specify which fields to check, what the check(s) are, and what action to take.

  3. The Alation APIs are used to trigger an action if conditions are met. Actions might include notify a catalog user in a conversation, or attach a policy to a catalog asset, for example.

A properly configured Bot has 1) an Alation Analytics sql query, 2) the rule(s), and 3) the action. When the rule’s conditions are met, the action is triggered.

After the Bot runs, the Bot’s Policy page is updated in the catalog showing when the Bot ran, how many objects were processed, the number that passed and the number of violations. The user can choose to view all objects in violation.

Here are two example use cases that can be configured as Bots:

  • Condition - “Find all table object pages in Alation that do not have a description”

    • Action - “Notify the data steward of violations so they can resolve them”

  • Condition - “Column names having certain keywords must have the PII Policy attached”

    • Action - “Attach the PII Policy to the column if not already attached”

  • Condition - “Check all schema object pages have a steward and belong to a domain”

    • Action - “Send a Slack message with Subject Schema Completeness to user_x@alation.com on the Slack channel Data Governance”

Notifications

You can choose to have a Bot send a notification as its action. A notification can be sent on one of three ways:

  1. Conversation: This uses the inbox and messaging feature within the Alation catalog

  2. Email: This uses the optional SMTP email server that can be configured in ASM’s settings page.

  3. Slack: This uses the optional Slack app that can be configured in ASM’s settings page.

Notifications are configured in ASM’s settings page.