Terraform Cloud Workspace Run Fails with Too Many Open Files

Configuration

• Terraform Cloud workspace Execution Mode is set to remote.

Problem

The Terraform Cloud workspace run fails and the run log will report the following:

Invalid value for "path" parameter: open
...
too many open files.

Cause

Substantial code can cause a run to breach the resources of the Terraform Cloud worker container, impacting the concurrent processes. If the workspace Execution Modeis configured to use remote operation, the runs are executed in the Terraform Cloud worker container. This worker container is a basic VM with limited resources regarding CPU and Memory.

Solutions

There are two distinct options to get around this error.  Option 1 is to limit the number of concurrent processes by adding the TFE_PARALLELISMenvironment variable within the Terraform Cloud Workspace or Option 2 suggests expanding the resource allocation for the workspace executions with the use of Terraform Agents. One of the benefits of Terraform Cloud Agents is Agents will run on User provision VMs to allow for adjustable resource allocation.  For more information please see the Agents documentation. 

NOTE: The TFE_PARALLELISM environment variable controls the number of concurrent operations that Terraform Cloud is executing. The default is 10.  For more information on the parallelism setting, please see the documentation on Special environment variables

Option 1 to add theTFE_PARALLELISMenvironment variable in the Workspace, follow the instructions below:

1. Go to https://app.terraform.io/app/<organization-name>/workspaces/<workspace-name>/variables and under Environment Variables click add.
2. Enter TFE_PARALLELISMin the Key field and under the Value field enter 1.

   NOTE: For large configurations decreasing the value to 1 may lead to a different issue - timeout of the plan phase. In case you hit a plan timeout, please increase the value of TFE_PARALLELISM until you find the best fit for your scenario. Unfortunately there is no unique formula providing opportunity to calculate the optimal value of TFE_PARALLELISM. 

3. Save the variable changes and conduct a rerun to verify if the run processes without the error.

Option 2 is to change the Workspace Execution Mode from Remote to Agent if Agents have been deployed within your environment. 

1. Go to https://app.terraform.io/app/<organization-name>/workspaces/<workspace-name>/settings/general  
2. Under General Settings locate the heading Execute Mode
3. Selection Agent using the radial button
4. Page to the bottom of the doc and click Save Settings
5. Retry the run and check the run log for the success status

Additional Information

• For additional assistance please contact HashiCorp Support to request additional assistance.