Problem
When the plan phase of a run completes in Terraform Enterprise, the run fails with an error. The final line of the plan output displays the following message:
failed to upload plan json: Bad status code: 403
Prerequisites
- Terraform Enterprise
Cause
After Terraform Enterprise completes a plan, it captures the generated plan file for use in the apply phase. To store this file, it makes API requests to its own endpoints using the instance's Fully Qualified Domain Name (FQDN).
This issue typically occurs when a network filtering device, such as a Web Application Firewall (WAF), inspects these internal requests and blocks them because they contain data that triggers a security rule.
Solutions
There are two primary solutions to resolve this issue.
Solution 1: Reconfigure Network Filtering Infrastructure
The recommended solution is to adjust your network filtering rules to allow this internal traffic.
To do this, inspect the logs from your filtering infrastructure (e.g., your WAF) to identify which rule is blocking the request from Terraform Enterprise. You can then reconfigure the rule to permit these requests or create an exception to allow the Terraform Enterprise instance to bypass the rule.
The exact steps required depend on the filtering service you use and are beyond the scope of this article.
Solution 2: Enable Hairpin Addressing
If you cannot modify the network filtering rules, you can enable hairpin addressing (also known as NAT loopback) within your Terraform Enterprise installation. This feature routes traffic from the Terraform Enterprise instance back to itself without sending it through external network infrastructure like a WAF.
Refer to the appropriate guide for your installation type for instructions:
- Standalone installations:Terraform Enterprise Hairpin Addressing Feature - Standalone
- Active/Active installations:Enable Hairpinning on Active-Active Terraform Enterprise
- Flexible Deployment Options (FDO) installations:How-to enable "hairpin_addressing" in Terraform Enterprise FDO Docker
Note that enabling hairpin addressing does not impact externally managed agents, as Terraform Enterprise does not manage their network traffic directly.
Outcome
After you reconfigure your network filter or enable hairpin addressing, subsequent Terraform plans should complete successfully without the 403 error.