How to alter agent worker pod specifications on Kubernetes/OpenShift – HashiCorp Help Center

Introduction

When you execute a run in a Terraform Enterprise workspace deployed on Kubernetes or OpenShift, Terraform Enterprise launches an agent worker in a new pod within a designated namespace. You can customize the specifications of this agent worker pod, such as its labels or resource limits.

This guide details two methods for altering the agent worker pod specification.

Expected Outcome

You will be able to launch an agent worker pod with custom labels and resource limits.

Prerequisites

An active installation of Terraform Enterprise running on Kubernetes or OpenShift.

Procedure

There are two methods to define custom pod settings. Choose the option that best fits your deployment strategy.

Option 1: Modify the Helm Chart

You can define the pod template directly in your Helm chart values.yaml file by adding an agentWorkerPodTemplate block. This method is suitable for static configurations managed with Helm.

Add the following example configuration to your Helm chart values. Adjust the labels and resource requests and limits as needed.

agentWorkerPodTemplate:
  metadata:
    labels:
      info: pod-template-info-hashicorp
  spec:
    containers:
      - resources:
          requests:
            memory: "2000Mi"
            cpu: "1000m"
          limits:
            memory: "4000Mi"
            cpu: "1500m"

Apply the updated Helm chart to your Terraform Enterprise installation.

Option 2: Use an Environment Variable

You can use the TFE_RUN_PIPELINE_KUBERNETES_POD_TEMPLATE environment variable to provide a base64-encoded JSON object containing the pod specification. This method is useful for dynamic configurations or when you cannot modify the Helm chart directly.

Create a JSON file named agent_pod.json with your desired pod specification.

{
  "metadata": {
    "labels": {
      "info": "pod-template-info-hashicorp"
    }
  },
  "spec": {
    "containers": [
      {
        "resources": {
          "requests": {
            "memory": "2000Mi",
            "cpu": "1000m"
          },
          "limits": {
            "memory": "4000Mi",
            "cpu": "1500m"
          }
        }
      }
    ]
  }
}

Encode the content of the JSON file into a single-line base64 string.

$ cat agent_pod.json | base64
## Output
ewogICJtZXRhZGF0YSI6IHsKICAgICJsYWJlbHMiOiB7CiAgICAgICJpbmZvIjogInBvZC10ZW1wbGF0ZS1pbmZvLWhhc2hpY29ycCIKICAgIH0KICB9LAogICJzcGVjIjogewogICAgImNvbnRhaW5lcnMiOiBbCiAgICAgIHsKICAgICAgICAicmVzb3VyY2VzIjogewogICAgICAgICAgInJlcXVlc3RzIjogewogICAgICAgICAgICAibWVtb3J5IjogIjIwMDBNaSIsCiAgICAgICAgICAgICJjcHUiOiAiMTAwMG0iCiAgICAgICAgICB9LAogICAgICAgICAgImxpbWl0cyI6IHsKICAgICAgICAgICAgIm1lbW9yeSI6ICI0MDAwTWkiLAogICAgICAgICAgICAiY3B1IjogIjE1MDBtIgogICAgICAgICAgfQogICAgICAgIH0KICAgICAgfQogICAgXQogIH0KfQ==

Set the TFE_RUN_PIPELINE_KUBERNETES_POD_TEMPLATE environment variable in your Terraform Enterprise deployment configuration using the base64-encoded string from the previous step.

TFE_RUN_PIPELINE_KUBERNETES_POD_TEMPLATE: ewogICJtZXRhZGF0YSI6IHsKICAgICJsYWJlbHMiOiB7CiAgICAgICJpbmZvIjogInBvZC10ZW1wbGF0ZS1pbmZvLWhhc2hpY29ycCIKICAgIH0KICB9LAogICJzcGVjIjogewogICAgImNvbnRhaW5lcnMiOiBbCiAgICAgIHsKICAgICAgICAicmVzb3VyY2VzIjogewogICAgICAgICAgInJlcXVlc3RzIjogewogICAgICAgICAgICAibWVtb3J5IjogIjIwMDBNaSIsCiAgICAgICAgICAgICJjcHUiOiAiMTAwMG0iCiAgICAgICAgICB9LAogICAgICAgICAgImxpbWl0cyI6IHsKICAgICAgICAgICAgIm1lbW9yeSI6ICI0MDAwTWkiLAogICAgICAgICAgICAiY3B1IjogIjE1MDBtIgogICAgICAgICAgfQogICAgICAgIH0KICAgICAgfQogICAgXQogIH0KfQ==

Verification

After applying one of the configurations, start a run in any workspace. Then, use kubectl to describe the newly created agent worker pod and verify that your custom settings have been applied.

$ kubectl -n terraform-enterprise-agents describe pods <your-worker-pod-name>

## ...
## Limits:
##   cpu: 1500m
##   memory: 4000Mi
## Requests:
##   cpu: 1
##   memory: 2000Mi
## ...

Additional Information

For more details on Helm chart configuration, refer to the official Terraform Enterprise Helm chart repository.
For more information on the environment variable, see the TFE_RUN_PIPELINE_KUBERNETES_POD_TEMPLATE documentation.