Introduction
You may need to adjust the TFE_PARALLELISM environment variable in Terraform Enterprise or HCP Terraform when infrastructure providers produce errors during concurrent operations or enforce non-standard rate limiting.
This article outlines several approaches for managing provider rate limits and improving run performance.
Recommendation
Success with custom values for TFE_PARALLELISM is highly dependent on your specific use case and infrastructure provider. Before making changes, review the following approaches to determine the best strategy for your environment.
Approach 1: Adjust TFE_PARALLELISM with Caution
The default value for TFE_PARALLELISM is 10. The primary consideration when changing this value is the API rate limiting on the provider's service (for example, the AWS API). If you encounter rate limits, Terraform runs may take longer than they would with the default parallelism setting.
In general, changing TFE_PARALLELISM alone may not significantly decrease the time a large run takes. Performance depends on the composition of your resources and the specific API calls required to manage them.
Approach 2: Restructure Configurations
To help decrease Terraform run times, consider breaking a large configuration into smaller, logically organized workspaces. This can reduce the number of concurrent operations within a single run and help avoid provider rate limits.
Approach 3: Use Multiple Provider Accounts
The most effective time reductions often come from sidestepping API rate limiting issues entirely. For instance, you can create multiple AWS accounts for your Terraform runs, as each account has an independent API call quota.
You can leverage multiple provider accounts within a single Terraform configuration to benefit from separate API call quotas, which makes it less likely that your API calls will be throttled. This architectural approach often yields greater performance gains than adjusting the parallelism setting.
Additional Information
- For more details on how Terraform processes resources concurrently, refer to the documentation on Terraform's Graph Walking Mechanism.