Introduction
Problem
In Terraform Enterprise (TFE) FDO v1.1.0, Terraform runs using remote agents may remain in a pending state and ultimately fail with x509 certificate verification errors. This prevents agents from registering with Terraform Enterprise and blocks all remote-agent–based runs.
{"@level":"info","@message":"2025-11-17T14:18:00.068Z [ERROR] agent: Failed starting core plugin: error=\"failed configuring core: agent registration failed: POST https://ccoe-tf-ent.hii-tsd.com/api/agent/register giving up after 1 attempt(s): Post \\\"https://ccoe-tf-ent.hii-tsd.com/api/agent/register\\\": tls: failed to verify certificate: x509: certificate signed by unknown authority\"","@module":"task-worker.executor.task-output","@timestamp":"2025-11-17T14:18:00.069218Z","id":"b5cc9a20-3c2a-49f5-8402-fceb7c2fc658","name":"agent-run","organization_name":"CCOE","run_id":"run-EGoATLGF5wvomhEp","stream":"stdout","workload_id":"plan-iSybzafGctwvx2ju","workload_type":"Plan","workspace_name":"aws-security-comm"}The issue is caused by incorrect file permissions on the CA certificate bundle inside the agent container, introduced by changes to the agent image build process in v1.1.0.Prerequisites
Terraform Enterprise FDO v1.1.0
1.1.1 using a CA Bundle
Remote agent workflows (VCS-driven runs confirmed)
Environments using self-signed or custom/internal CAs are most impacted
No runs are executed using remote agentsOverview of possible solutions (if applicable)
Cause
- This is caused by a bug in which permissions of the CA certificates file are incorrect.
Solutions
As a short-term supported workaround, the agent image can be manually rebuilt using the Docker CLI to preserve the correct file permissions on the CA certificate bundle. This is done by executing a Docker build from within the Terraform Enterprise container, which rebuilds the hashicorp/tfe-agent:now image using the existing task-worker context. While this approach can immediately restore agent registration and allow runs to proceed, it has important limitations: the rebuild must be repeated any time the agent image is regenerated, and it is not considered a long-term or permanent solution. A permanent solution has been implemented in a GitHub Pull Request 3414.. Customers should upgrade to a patched release once available. Below is the curl command showing how to rebuild the container as a workaround.
docker exec -u0 -ti "terraform-enterprise-tfe-1" sh -c 'tar -C /run/terraform-enterprise/task-worker/docker -cf - . | curl \
--unix-socket /var/run/docker.sock \
-H "Content-Type: application/x-tar" \
--data-binary @- \
"http:/v1.43/build?t=hashicorp/tfe-agent:now"Outcome
The fix restores correct permissions on the CA bundle inside the agent image
Additional Information
- Here is a link to the Release notes for the patch version:
1.1.x - Terraform Enterprise | Terraform | HashiCorp Developer