Access Secrets Manager from Alation Agent Using an IAM Instance Profile

Alation Cloud Service Applies to Alation Cloud Service instances of Alation

Use the information in this topic to configure an Alation Agent installed on an Amazon EC2 instance to read secrets from AWS Secrets Manager using an IAM instance profile. With the Alation Agent, AWS Secrets Manager, and your on-premise data source all running within your Virtual Private Cloud (VPC), your data source credentials will never leave your VPC, and they are not stored in Alation or the Alation Agent.

Note

To see other options for integrating with AWS Secrets Manager, see Integration with AWS Secrets Manager for Data Source Authentication Using OCF Connector.

Prerequisites

To use this feature, you must meet the following requirements:

  • You must be an Alation Cloud Service customer on the cloud-native architecture.

  • You must be using Alation Agent version 1.7.2.4360 or later. For help installing the Alation Agent, see Install the Alation Agent.

  • To access AWS Secrets Manager using an IAM instance profile:

    • You must be on Alation Cloud Service version 2024.1.4 or later.

    • Your Alation Agent must be installed on an Amazon EC2 instance.

    • You must install Authentication Service Add-on version 5.14.0.1882 or later on the Alation Agent.

  • To access AWS Secrets Manager using an IAM user’s credentials:

    • You must be on Alation Cloud Service version 2024.1.5 or later.

    • You must install Authentication Service Add-on version 5.14.0.1968 or later on the Alation Agent.

  • You must have a supporting OCF connector installed on the Agent. To find out if a specific connector supports this feature, see the documentation for the specific connector in the Open Connector Framework (OCF) Overview section.

    Note

    You can use the Native Data Sources API to migrate a data source from an OCF connector that’s not on an Alation Agent to an OCF connector that is on an Alation Agent.

Step 1: Create a Security Policy to Access Secrets Manager

This step is performed in the AWS IAM console

For the AWS Secrets Manager integration, you’ll need an AWS IAM policy granting access to AWS Secrets Manager. Here and below, a substitute name read_secrets_policy is used to refer to this policy.

If you haven’t established a read_secrets_policy yet, create one:

  1. Log in to the AWS IAM management console.

  2. From the left-hand menu, under Access management, select Policies.

  3. Click Create policy. The policy editor will open.

  4. Under Select a service, type secrets in the search bar and select Secrets Manager. This will display actions relevant to the service.

  5. Under Actions allowed, expand the Read list and select the checkbox for the action GetSecretValue.

  6. Under Resources, select All.

    ../../../../_images/Configure_Secrets_Manager_Create_Policy.png

    Note

    You can restrict the permissions by providing access to specific resources only:

    1. Instead of All, select Specific.

    2. Click Add ARNs to specify the resources by entering the resource’s region, ARN (Amazon Resource Name), and secret.

    You can adjust the resource permissions anytime by editing this policy.

  7. Click Next. The Review and create screen will open.

  8. Under Policy details, in the Policy name field, provide a meaningful name.

  9. Review the details, and click Create policy to create the policy. It will be added to the list of policies under your AWS account. Your policy JSON will look similar to the following:

    {
  "Version": "2012-10-17",
  "Statement": [
    {
        "Sid": "VisualEditor0",
        "Effect": "Allow",
        "Action": "secretsmanager:GetSecretValue",
        "Resource": "*"
    }
  ]
}

Note

An Alation Agent deployed on an AWS EC2 machine can retrieve secrets from a separate AWS account (cross-account access). For example, the Alation Agent machine could be deployed in Account 1, and the AWS Secrets Manager could be deployed in Account 2. This requires additional setup as described in AWS documentation here: Access AWS Secrets Manager secrets from a different account.

Step 2: Create an AWS IAM Role for the Alation Agent’s EC2 Instance

This step is performed in the AWS IAM console

Now that you’ve created an IAM policy for accessing AWS Secrets Manager, you need to create an AWS IAM role and attach the policy to it. Here and below, a substitute name read_secrets_role is used to refer to this role.

If you haven’t established a read_secrets_role yet, create one:

  1. From the left-hand menu, under Access management, select Roles.

  2. Click Create role. The role editor will open.

  3. Leave AWS service as the Trusted entity type (default) and select EC2 as the Use case.

    ../../../../_images/Configure_Secrets_Manager_Create_Role.png

  4. Click Next. The Add permissions screen will open.

  5. Under Permissions policies, search for the policy you created to allow reading secrets from Secrets Manager. When found, select the checkbox of this policy in the policies table.

  6. Click Next. The Name, review, and create screen will open.

  7. In the Role name field, provide a meaningful name. We’ll use read_secrets_role as example.

  8. In the Description field, provide a description.

  9. Review the role information and click Create role to create the role. It will be added to the list of roles under your AWS account. This role will be assumed by the instance profile to access Secrets Manager from Alation.

  10. Attach this role to the EC2 instance where the Alation Agent is running: navigate to EC2, open the properties of the instance, and select Action > Security > Modify IAM Role > Select IAM Role > Update IAM Role. For more information, refer to Attach an IAM role to an instance in AWS documentation.

Step 3: Install the Authentication Service Add-on on the Alation Agent

This step is performed on the Alation Agent machine

To enable the Alation Agent to communicate with AWS Secrets Manager, you must install the Authentication Service add-on on the Agent.

To install or update the Authentication Service add-on on an Alation Agent:

  1. If you don’t have the latest plugin yet, contact Alation Support to request it. The plugin is a Docker image named auth-service-docker-image-<plugin-version>tar.gz.

  2. If needed, transfer the downloaded file to the Agent’s host machine. For example, if you downloaded the plugin file to a Unix-based machine, you could transfer the file using the scp command in Terminal:

    scp /local/path/to/auth-service-docker-image-<plugin-version>.tar.gz <ssh-user>@<server-address>:/remote/path/to/agent

  3. On the Agent’s host machine, unzip the .tar.gz file. Example:

    gzip -d auth-service-docker-image-<plugin-version>.tar.gz

  4. Install or update the plugin. To install the plugin from scratch, use this command:

    sudo kratos addons install auth ./auth-service-docker-image-<plugin-version>.tar

    To update the plugin, use this command:

    sudo kratos addons update auth ./auth-service-docker-image-<plugin-version>.tar

  5. Restart the Agent:

    sudo systemctl restart hydra

  6. Check that the plugin is running. It may take about two minutes for the plugin to start up.

    sudo docker ps

    In the output of this command, you should see auth listed under IMAGE, and under STATUS it should indicate that the plugin is Up. For example:

    CONTAINER ID   IMAGE                 COMMAND                  CREATED          STATUS                             PORTS                                           NAMES
00d929b2582b   auth                  "java -Dlog4j.config…"   11 seconds ago   Up 10 seconds                      0.0.0.0:11001->11001/tcp, :::11001->11001/tcp   auth
f80e23b27e2a   application_gateway   "/opt/cag"               11 seconds ago   Up 10 seconds (health: starting)                                                   cag
c8c16128644e   proxy                 "/opt/reverseProxy"      12 seconds ago   Up 11 seconds (health: starting)                                                   proxy
8c3d5cfeb3fd   connector_21          "/opt/entrypoint.sh …"   12 seconds ago   Up 12 seconds (health: starting)   127.0.0.1:10021->10021/tcp                      connector21
c8814bcadc3c   agent                 "/opt/agent"             13 seconds ago   Up 12 seconds (health: starting)   127.0.0.1:8080->8080/tcp                        agent

The Authentication Service add-on has now been installed. To troubleshoot the Agent or Authentication Service add-on, see Troubleshoot the Agent.

Step 4: Create an Authentication Profile

This step is performed in Alation

To create an authentication profile for the Secrets Manager integration:

  1. Log in to your Alation instance as a Server Admin.

  2. Click the Admin Settings gear icon on top right to open the Admin Settings page.

  3. Click Authentication to open the Authentication tab. Locate the section Authentication Configuration Methods for External Systems.

  4. Next to See configurations for, click the drop-down menu and select the relevant Alation Agent.

  5. Click Add Configuration, and then select AWS Secrets Manager as the method type. If you’re creating a configuration for an Alation Agent, the only option is AWS Secrets Manager. The Authentication Configuration Method page will open in a new browser tab.

  6. In Config Name, enter a unique name for the configuration. Save it for future reference when configuring the data source.

  7. Under Region, select the appropriate AWS region for the Secrets Manager service (the region under which your secrets are stored).

  8. Under Authentication Type, select IAM Instance Profile.

  9. Click Save. Alation attempts to create a connection, and if the connection is successful, the configuration is saved.

Now, you can use your integration with an OCF connector. See next: Configure Authentication with AWS Secrets Manager in Data Source Settings.