Terraform Enterprise Fails to Start when using Azure Redis with MSI

Introduction

Terraform Enterprise supports connecting to an Azure Redis instance using Managed Service Identity (MSI) authentication. This method enhances security by eliminating the need for static credentials. You can find the official documentation on how to connect to Redis using MSI authentication.

Problem

After configuring Terraform Enterprise to use Redis with MSI authentication, the application fails to start. This indicates a potential issue with the MSI authentication configuration in Azure or the environment variables within the Terraform Enterprise container.

Prerequisites

• Terraform Enterprise version 1.0.0 or newer.
• An operational Terraform Enterprise environment deployed in Azure.

Cause

The startup failure may be caused by an incorrect MSI authentication configuration in Azure, preventing Terraform Enterprise from obtaining the necessary access token to connect to Redis.

Solution

Verify the MSI Authentication Configuration

Follow these steps to connect to the Terraform Enterprise container and manually test the Redis connection. This procedure helps isolate whether the issue is with the Azure configuration or with Terraform Enterprise itself.

1. Connect to the Terraform Enterprise container. Choose the command that matches your environment.

   For Docker environments, connect to the container.

   # docker exec -it <terraform_enterprise_container> bash

   For Kubernetes environments, connect to the pod.

   # kubectl -n terraform-enterprise exec -it <terraform_enterprise_pod> -- bash

2. Verify that the Redis environment variables are correctly set inside the container.

   # env | grep REDIS

   The output should display your configuration values.

   TFE_REDIS_USE_AUTH=false
   TFE_REDIS_PASSWORDLESS_AZURE_CLIENT_ID=<client_id>
   TFE_REDIS_HOST=<redis_fqdn>:6380
   TFE_REDIS_USER=<principal_id>
   TFE_REDIS_PASSWORDLESS_AZURE_USE_MSI=true
   TFE_REDIS_USE_TLS=true
   TFE_REDIS_SIDEKIQ_PASSWORDLESS_AZURE_USE_MSI=true

3. Request an MSI authentication token for Redis from the Azure metadata service. Replace <CLIENT_ID> with your actual client ID.

   # curl -H "Metadata: true" \
     "http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https://redis.azure.com/&client_id=<CLIENT_ID>" | jq -r .access_token

   The command returns an access token. You will use this token in a later step.

   eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IkhTMj...##

4. Connect to the Redis instance using redis-cli.

   # redis-cli -h <redis_fqdn> -p 6380 --tls

   A successful connection displays the Redis prompt.

   <redis_fqdn>:6380>

5. Authenticate to Redis using your principal_id and the access token obtained in step 3.

   <redis_fqdn>:6380> AUTH <principal_id> <TOKEN_FROM_PREVIOUS_STEP>
   OK

6. Test the authenticated connection with the PING command.

   <redis_fqdn>:6380> ping
   PONG

Outcome

A PONG response confirms that the Azure MSI authentication configuration is correct and that the Terraform Enterprise container can successfully connect and authenticate to Redis.

If any of these steps fail, review your Azure configuration with your Azure administrator team to resolve the underlying issue.

Additional Information

• Official Documentation for Redis Configuration in Terraform Enterprise