TF logging in Terraform Enterprise showing [TRACE] level logging despite not being set.

Problem

TF logging in Terraform Enterprise showing [TRACE] level logging in certain runs, despite not being set.

Cause

• [TRACE] level logging in TF runs in TFE occur if the run consumes more RAM than is allocated for TF isolation containers in TFE's Administrative Settings.

Solutions:

• Split the workspace into separate smaller workspaces to consume less memory per run.

• Refactor the .tf code to use provider aliases as this will optimize the memory consumed by provider plugins.

• Terraform CLI 0.12.29 and above can play significant role in decreasing memory footprint due to the change on how it stores resource graph objects. If you are not already on this version of Terraform CLI, HashiCorp Support encourages you to upgrade Terraform CLI 0.12.29 and above when possible.

• Increase memory capacity of worker container where upgrade is not an option or not resolving the symptom. The maximum memory per run is set in the installer dashboard at https://TFE_HOSTNAME:8800 under Settings > Capacity. Experimenting with this setting by incrementally increasing the memory allocation setting may be necessary if the initial change does not resolve the issue.

• When increases the maximum memory per run, the total memory required for the host machine to operate Terraform Enterprise will also increase as side effect, the memory capacity of the host machine should also be adjusted in accordance with the guidelines.

Additional Information

• <Hashicorp KB on troubleshooting Plan Operation Failures>