Atlas Fails to Start in Terraform Enterprise Consolidated Services Mode – HashiCorp Help Center

Problem

Users of Terraform Enterprise in consolidated services mode may encounter a critical issue during Atlas startup. This problem relates to the absence of an essential environment variable, AMPQ_URL, in the terraform-enterprise container. While this might not be an issue for most users, it can lead to complications for individuals with specific network configurations.

Prerequisites

Replicated Deployment of Terraform Enterprise v202309-1 in consolidated services mode
May also affect Docker Flexible Deployment Option installations of Terraform Enterprise

Cause

In Terraform Enterprise's consolidated services mode, the AMPQ_URL environment variable is not set in the terraform-enterprise container. This omission is typically inconsequential because there is no longer a RabbitMQ service in TFE. However, Atlas, the component responsible for certain operations, attempts to connect to rabbitmq:5672 by default during startup. This behavior differs from previous versions, where it attempted to connect to tfe-rabbitmq:5672.

For standard TFE installations, the absence of either hostname (rabbitmq or tfe-rabbitmq) is generally harmless, as a DNS resolution error only emits a warning and Atlas proceeds to start up. However, for users who run a private RabbitMQ service on their corporate network, which often uses a specific naming convention (e.g., rabbitmq.mycompany.com) with DNS configuration set to search in mycompany.com, this can result in a non-recoverable authentication error.

ERROR -- #Bunny::Session:0x1f5e0 hashicorp@rabbitmq:5672, vhost=hashicorp, 
addresses=[rabbitmq:5672]&gt;: Authentication with RabbitMQ failed: 403 
ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. 
For details see the broker logfile.","component":"atlas"}

Solution

Users facing the issue of Atlas failing to start in Terraform Enterprise consolidated services mode due to the absence of the AMPQ_URL environment variable can consider the following solution:

Remove the Search Entry in /etc/resolv.conf: A viable workaround is to modify the /etc/resolv.conf file on your TFE Host, to eliminate the search entry that resolves to the problematic hostname. This solution may be suitable for users, especially if there are no additional services on the host relying on that specific DNS configuration. However, please note that it might not be effective for all scenarios.

Outcome

Terraform Enterprise Engineering has been alerted to this issue and is currently in the process of investigating a resolution that will be included in a later release of Terraform Enterprise.

Additional Information

If you have additional questions or concerns, please contact HashiCorp Support.