Problem
When using Terraform Enterprise (TFE) Flexible Deployment Options (FDO) on Kubernetes, runs for workspaces with names longer than 63 characters fail and become stuck in the Plan Queued stage.
The task-worker.log displays the following error, indicating that the workspace name exceeds the character limit for Kubernetes metadata labels.
{
"@level": "error",
"@message": "error running task instance",
"@module": "task-worker.executor",
"err": "error creating kubernetes job: Job.batch \"tfe-task-32428532-468d-4b0e-b065-ccfa77df0271\" is invalid: [metadata.labels: Invalid value: \"this-workspace-has-a-name-with-70-characters-which-is-causing-an-issue\": must be no more than 63 characters, spec.template.labels: Invalid value: \"this-workspace-has-a-name-with-70-characters-which-is-causing-an-issue\": must be no more than 63 characters]"
}Prerequisites
- Terraform Enterprise Flexible Deployment Options deployed on Kubernetes.
- A workspace with a name longer than 63 characters.
This issue affects all current versions of TFE FDO for Kubernetes.
Cause
This issue occurs because Kubernetes imposes a 63-character limit on job labels. Terraform Enterprise uses the workspace name to generate these labels, causing an error when the name is too long.
Solutions
There are two primary solutions to resolve this issue.
Solution 1: Offload the Workspace to an Agent Pool
You can configure the workspace to execute its runs on a self-hosted agent pool. This approach bypasses the Kubernetes job creation process on the main TFE application pods, avoiding the label length limitation.
Solution 2: Rename the Workspace
Rename the workspace to have a name with 63 or fewer characters. You can do this through the UI or the API.
Using the UI
- Navigate to the General Settings of the affected workspace.
- Update the workspace name in the Name field.
- Click Save settings at the bottom of the page.
Using the API
-
Create a file named
payload.jsonwith the following content. Replace the placeholder with the new workspace name.{ "data": { "attributes": { "name": "new-workspace-name-under-63-characters" } } }Specifying only the
nameattribute ensures that no other workspace settings are modified. -
Export an API token as an environment variable. Replace
<your_token_here>with your actual token.$ export TOKEN=<your_token_here>
-
From the directory containing
payload.json, run the followingcurlcommand to rename your workspace. Replace<your_workspace_id_here>with the actual workspace ID.$ curl \ --header "Authorization: Bearer $TOKEN" \ --header "Content-Type: application/vnd.api+json" \ --request PATCH \ --data @payload.json \ https://fdo-k8s-latest.xxx.com/api/v2/workspaces/<your_workspace_id_here>
Outcome
After applying one of the solutions, runs in the workspace should execute successfully without getting stuck in the Plan Queued stage.