How to alter agent worker pod specifications on Kubernetes/OpenShift

Introduction

With Terraform Enterprise on Kubernetes/Openshift when you start a run on a workspace it will cause Terraform Enterprise to start an agent worker on a new pod in a different namespace. 

There is the ability to alter the pod settings that will be started by Terraform Enterprise. 

Expected Outcome

Have the agent worker pod with different labels and limits. 

Prerequisites

•  Terraform Enterprise running on Kubernetes or Openshift. 

Use Case

There are 2 options to define this. Please use one of these options 

Option 1:

Alter the helm chart to add the following. This is just an example

agentWorkerPodTemplate:
  metadata:
    labels:
      info: pod-template-info-hashicorp
  spec:
    containers:
    - resources:
        requests:
          memory: "2000Mi"
          cpu: "1000m"
        limits:
          memory: "4000Mi"
          cpu: "1500m" 

The official documentation for the helm release can be found here

Option 2:

Use the environment parameter TFE_RUN_PIPELINE_KUBERNETES_POD_TEMPLATE as documented here 

• Create the pod specification as json

{
  "metadata": {
    "labels": {
      "info": "pod-template-info-hashicorp"
  }
},
  "spec": {
    "containers": [
      {
        "resources": {
          "requests": {
            "memory": "2000Mi",
            "cpu": "1000m"
          },
        "limits": {
           "memory": "4000Mi",
           "cpu": "1500m"
        }
       }
     }
   ]
  }
}

• Make this an encoded string

cat agent_pod.json | base64

ewogICJtZXRhZGF0YSI6IHsKICAgICJsYWJlbHMiOiB7CiAgICAgICJpbmZvIjogInBvZC10ZW1wbGF0ZS1pbmZvLWhhc2hpY29ycCIKICAgIH0KICB9LAogICJzcGVjIjogewogICAgImNvbnRhaW5lcnMiOiBbCiAgICAgIHsKICAgICAgICAicmVzb3VyY2VzIjogewogICAgICAgICAgInJlcXVlc3RzIjogewogICAgICAgICAgICAibWVtb3J5IjogIjIwMDBNaSIsCiAgICAgICAgICAgICJjcHUiOiAiMTAwMG0iCiAgICAgICAgICB9LAogICAgICAgICAgImxpbWl0cyI6IHsKICAgICAgICAgICAgIm1lbW9yeSI6ICI0MDAwTWkiLAogICAgICAgICAgICAiY3B1IjogIjE1MDBtIgogICAgICAgICAgfQogICAgICAgIH0KICAgICAgfQogICAgXQogIH0KfQ==

• Add the TFE_RUN_PIPELINE_KUBERNETES_POD_TEMPLATE as an environment variable

 TFE_RUN_PIPELINE_KUBERNETES_POD_TEMPLATE: ewogICJtZXRhZGF0YSI6IHsKICAgICJsYWJlbHMiOiB7CiAgICAgICJpbmZvIjogInBvZC10ZW1wbGF0ZS1pbmZvLWhhc2hpY29ycCIKICAgIH0KICB9LAogICJzcGVjIjogewogICAgImNvbnRhaW5lcnMiOiBbCiAgICAgIHsKICAgICAgICAicmVzb3VyY2VzIjogewogICAgICAgICAgInJlcXVlc3RzIjogewogICAgICAgICAgICAibWVtb3J5IjogIjIwMDBNaSIsCiAgICAgICAgICAgICJjcHUiOiAiMTAwMG0iCiAgICAgICAgICB9LAogICAgICAgICAgImxpbWl0cyI6IHsKICAgICAgICAgICAgIm1lbW9yeSI6ICI0MDAwTWkiLAogICAgICAgICAgICAiY3B1IjogIjE1MDBtIgogICAgICAgICAgfQogICAgICAgIH0KICAgICAgfQogICAgXQogIH0KfQ==

Verifying the result:

• Start a run on a workspace and describe the worker pod to see the details
  
  

kubectl -n terraform-enterprise-agents describe pods

...
...
Limits:
  cpu: 1500m
  memory: 4000Mi
Requests:
  cpu: 1
  memory: 2000Mi

Additional Information

• Terraform Enterprise Helm chart can be found here

• Details about the parameter TFE_RUN_PIPELINE_KUBERNETES_POD_TEMPLATE can be found here