How to create a custom Terraform Build Worker image

Expected Outcome

Use a custom TBW image with Terraform Enterprise

Prerequisites

• Terraform Enterprise until v202308-1
• Terraform Enterprise v202302-1 through v202308-1 require run_pipeline_mode to be set to legacy

Use Case

• If your organizations Terraform Configurations are expected to use additional tools not available in the default image, ex: azure cli, kubernetes cli, CA certificates. Creating a custom Terraform Build Worker image will be desired.

Procedure

• This guide uses the default terraform worker image as the base for the custom image. Please check the image history first (use --no-trunc for full output):                                  

  $ docker image history hashicorp/build-worker:now
  IMAGE CREATED CREATED BY SIZE COMMENT
  71b2010fd6a9 4 days ago /bin/sh -c #(nop) ADD file:173252116b3482470… 221kB
  c9d9dc231b2e 4 days ago /bin/sh -c #(nop) LABEL com.hashicorp.conta… 0B
  c0eae83ebcae 2 weeks ago /bin/sh -c #(nop) COPY file:da5887adb3c61bb2… 1.94MB
  ...... truncated output

• Next step is to tag, login and push the base image to a registry (Skip if not using a Docker registry).
  

  # Tag the image before pushing it to the repository
  $ docker tag hashicorp/build-worker:now custom/tbw:core
  
  # Verify the image is tagged
  $ docker image ls custom/tbw:core
  REPOSITORY TAG IMAGE ID CREATED SIZE
  jsorondo/tbw core 71b2010fd6a9 4 days ago 316MB
  
  # Log into your docker registry (for self-hosted registry add host:port)
  $ docker login -u $USERNAME
  ...... 
  Login Succeeded
  
  # Push the base image to the registry
  $ docker push custom/tbw:core
  The push refers to repository [docker.io/jsorondo/tbw]
  e9ba7e467bc1: Pushed
  ......
  69f57fbceb1b: Pushed
  core: digest: sha256:199ee2b1c85bb72965ae69611c107db8f4632f95125d03e14ef1c70ae9648a64 size: 1791

                                                                           
• Create a file called Dockerfile and customize it as needed. Please keep in mind that the final size of the image will impact performance.      
  

  # This Dockerfile builds the image used for the worker containers.
  FROM custom/tbw:core
  
  # Install required software for Terraform Enterprise.
  RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
      apt-transport-https lsb-release gnupg
  RUN curl -sL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > /etc/apt/trusted.gpg.d/microsoft.asc.gpg && \
      CLI_REPO=$(lsb_release -cs) && \
      echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ ${CLI_REPO} main" \
      > /etc/apt/sources.list.d/azure-cli.list && \
      curl -fsSLo /etc/apt/trusted.gpg.d/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && \
      echo "deb [signed-by=/etc/apt/trusted.gpg.d/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" \
      > /etc/apt/sources.list.d/kubernetes.list && \
      apt-get update && \
      apt-get install -y azure-cli kubectl && \
      rm -rf /var/lib/apt/lists/*
  
  # Include all necessary CA certificates.
  ADD example-root-ca.crt /usr/local/share/ca-certificates/
  ADD example-intermediate-ca.crt /usr/local/share/ca-certificates/
  
  # Update the CA certificates bundle to include newly added CA certificates.
  RUN update-ca-certificates

     
• Build the Docker image and push it to your registry.
  

  # Building the Docker image
  $ docker build -t custom/tbw:120522 .
  Sending build context to Docker daemon 47.62kB
  Step 1/4 : FROM custom/tbw:core
  ---> 71b2010fd6a9
  Step 2/4 : RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends apt-transport-https lsb-release gnupg
  ---> Running in 8e6ad61e6595
  Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
  Get:2 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
  Get:3 http://ppa.launchpad.net/git-core/ppa/ubuntu bionic InRelease [20.8 kB]
  Get:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
  ...... 
  
  # Pushing the Docker image to the Registry
  $ docker push custom/tbw:120522
  The push refers to repository [docker.io/custom/tbw]
  9ca38bfd21fb: Pushed
  1b89fa7ab6f0: Pushed
  8a67973f36a6: Pushed
  e9ba7e467bc1: Layer already exists
  ......
  120522: digest: sha256:328e1b593cd2a08b2242ebe3f4a60ff0f26169a0ff04bf8746bbefc0cb6e4ed7 size: 2426                                               

                                                                                                    
• Update the configuration in Terraform Enterprise to reference the custom worker image and restart the application.       

  $ replicatedctl app-config set custom_image_tag --value 'custom/tbw:120522'
  $ replicatedctl app-config set tbw_image --value 'custom_image'
  $ replicatedctl app apply-config

Additional Information

• Alternative Terraform worker image

• How to monitor Terraform worker resource usage