Introduction
In some scenarios, it may be useful to consider adjusting the TFE_PARALLELISM
environment variable.
Scenario
When infrastructure providers are producing errors on concurrent operations or use non-standard rate limiting. Consider changing `parallelism` in your Terraform Enterprise or Terraform Cloud backed workspaces.
Recommendation
Success with using custom values for TFE_PARALLELISM
is highly dependent on each individual case. The default value is 10. The primary consideration when changing this is rate limiting on the service side (e.g., the AWS API). If rate limits are encountered, runs can actually take more time than if the default parallelism value had not been altered.
Generally speaking, changing TFE_PARALLELISM
will not significantly decrease the time that a large run takes, though this will depend on the makeup of those resources and the API calls required to instantiate them and can vary.
To help decrease Terraform run times, it can help to break a configuration into smaller workspaces if it's possible to organize those resources in a logical, straightforward way.
That being said, the best net time reductions come about working to sidestep API rate limiting issues entirely.
For instance, in AWS you could create multiple AWS accounts for the runs in question, as each account has its own independent API call quota. Multiple accounts can be leveraged in a single configuration for Terraform runs and benefit from the separate API call quotas (and by consequence API calls are less likely to get throttled).
Tweaking Terraform itself is not likely to net the kind of performance gains that such an approach will yield.