Workspace Runs: Troubleshooting long wait times or slowness post plan output execution in Terraform Enterprise

Problem

In Terraform Enterprise, a run may experience an abnormally long wait time after the plan phase completes. This delay occurs before the next action becomes available, such as the Confirm & Apply option, or before the Cost Estimation, Sentinel, or apply phase begins.

Cause

This issue can be caused by one or both of the following factors:

1. Outdated Terraform Version: The workspace is configured to use a Terraform binary version older than v0.15.5. Optimizations to the run workflow were introduced in this version.
2. Inefficient Docker Storage Driver: The Docker storage driver configured for the Terraform Enterprise installation is not overlay2. Docker's official documentation recommends overlay2 for all currently supported Linux distributions for best performance.

Solutions

Solution 1: Upgrade the Workspace Terraform Version

Ensure the workspace is using Terraform v0.15.5 or a newer version.

1. Navigate to your workspace and select Settings > General.
2. In the Terraform Version dropdown menu, select v0.15.5 or a later version.
3. If v0.15.5 is not available, you may need to add it to your Terraform Enterprise installation by following the instructions in Adding Terraform Versions to Terraform Enterprise.

Note: If your workspace was previously using a version older than v0.15.0, you must review the official Upgrading to Terraform v0.15 guide to ensure your configuration is compatible before changing the version.

Solution 2: Configure the Docker Storage Driver

Update the Docker storage driver on your Terraform Enterprise host to overlay2.

1. Check the current storage driver by running the following command on the host.

   $ docker info

2. In the output, locate the Storage Driver line. If it is not overlay2, proceed with the next steps.

3. Stop the Terraform Enterprise application before making changes to the Docker configuration.

   Warning: Changing the storage driver makes existing containers and images inaccessible on the local system. Before proceeding, use docker save to back up any custom images or push them to a container registry.

4. Follow the official [Docker documentation on using the OverlayFS storage driver](https://support.hashicorp.com/Use%20the OverlayFS storage driver | Docker Documentation) to change the configured driver.
5. After the change is complete, restart the Terraform Enterprise application.

Outcome

After implementing one or both of these solutions, you should observe a significant decrease in the wait time between the plan and apply phases of a run.

Additional Information

• Docker storage drivers | Docker Documentation
• Upgrading to Terraform v0.15