Occasionally while refreshing state during runs, an error reading connection is shut down, or unexpected EOF may repeatedly appear until the run ultimately fails. Often, though not always, this is a sign that the Terraform runs are exceeding the currently configured maximum memory for the run containers, and need more memory allocated per run to complete successfully. Another indicator would be the error message cannot allocate memory appearing in the run output indicating the pre-allocated memory is insufficient and tuning should be done accordingly.
Resolving the issue
The maximum memory per run is set in the installer dashboard at https://TFE_HOSTNAME:8800 under Settings > Capacity. Experimenting with this setting by incrementally increasing the memory allocation setting may be necessary if the initial change does not resolve the issue.
Note that this setting is not limited by system checks; it depends on the operator to provide enough memory to the system to accommodate the requested limits. If the memory limit is adjusted to 512 MB with the default capacity of 10, the instance would require, at a minimum, 5.2 GB of memory reserved for Terraform runs. Additional information on managing run capacity and concurrency settings can be found in our Capacity and Performance document.
If you continue to experience issues after increasing the maximum memory, please contact HashiCorp support.
Resolving the issue
The maximum memory per run is set in the installer dashboard at https://TFE_HOSTNAME:8800 under Settings > Capacity. Experimenting with this setting by incrementally increasing the memory allocation setting may be necessary if the initial change does not resolve the issue.
Note that this setting is not limited by system checks; it depends on the operator to provide enough memory to the system to accommodate the requested limits. If the memory limit is adjusted to 512 MB with the default capacity of 10, the instance would require, at a minimum, 5.2 GB of memory reserved for Terraform runs. Additional information on managing run capacity and concurrency settings can be found in our Capacity and Performance document.
If you continue to experience issues after increasing the maximum memory, please contact HashiCorp support.