How to add AWS, GCP, and Azure CLI to your custom tfc-agent image

Avatar
Irving
  • Updated

Expected Outcome

This article outlines the details and process to install common cloud provider CLIs from AWS, Google Cloud, and Azure to Terraform Cloud/Enterprise agents.

Prerequisites

  • Docker
  • A text editor
  • Access to a Docker registry (i.e. Docker Hub, GHCR, Quay, Artifactory)

Use Case

  • Some Terraform workflows depend on having the CLI of a given cloud provider available. As one example, using aws eks get-token to authenticate with AWS's Elastic Kubernetes Service.

Concepts

Docker

TFC/E agents are based on the hashicorp/tfc-agent image found on Docker Hub: https://hub.docker.com/r/hashicorp/tfc-agent

All the examples below start from the same template Dockerfile:

FROM hashicorp/tfc-agent:latest

USER root

USER tfc-agent

A full reference on the Dockerfile format can be found in Docker's documentation: https://docs.docker.com/engine/reference/builder/

The FROM line defines the base container to start modifying from. The latest tag can be modified toreflect a version number as found in the image tags (https://hub.docker.com/r/hashicorp/tfc-agent/tags)

RUN lines (not in the template, but used later) are executed in a shell in the container and can be used to install software, edit configuration, etc.

The USER line defined which user should be made active for the following sections in a Dockerfile. In this template, any RUN instructions will be exectuted as root if the RUN line is placed after USER root line and before the `USER tfc-agent` line. After the USER tfc-agent line, any additional RUN lines and the container itself when deployed runs with the tfc-agent user. Ordering of these lines are important because the tfc-agent user won't have the required permissions to install most software.

Using the template, software can be added to the TFC/E agent by adding the correct RUN lines in the Dockerfile in order to install the software.

Building a Docker Image

With a Dockerfile, an image can be created by executing docker build . in the same directory as the Dockerfile (or by specifying a path to it with -f) and providing a context, such as the current directory, ..

Examples: docker build . or docker build -f ../otherdir/Dockerfile ../otherdir

Tagging Docker images

To tag a Docker image in order to push it to a registry, form a registry identifier with the following information: [registry]/<user>/<name>[:version] where:

- [registry] is the Docker registry to push to, which for Docker defaults to Docker Hub if omitted

- <user> which should be the username on the Docker registry

- <name> which should be the image name.

- [:version] which is the optional tag, prefixed with :. This defaults to latest

As an example, the base tfc-agent image used in this article would be docker.io/hashicorp/tfc-agent:latest

docker build takes in the -t argument to tag a Docker image. For example: docker build -t quay.io/myorg/myagent . or docker build -f ../otherdir/Dockerfile -t mydockerhuborg/myagent ../otherdir

Pushing Docker images

Given a registry identifier, called $REGISTRY_IMAGE in this article, the image can be pushed from a local machine to the appropriate registry by using docker push $REGISTRY_IMAGE. A docker login may be required to authenticate with the Docker registry provider. Refer to the registry documentation for authentication details.

Procedure (AWS)

For the AWS CLI, the instructions from https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html are followed

  • Create a file called Dockerfile which defines the custom image.
    FROM hashicorp/tfc-agent:latest

    USER root

    RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" \
        && unzip awscliv2.zip \
        && ./aws/install

    USER tfc-agent

    # This line will error on build if the AWS CLI is not installed
    RUN aws --version
  • While in the same directory as the Dockerfile, Build the image with docker build -t $REGISTRY_IMAGE . where the $REGISTRY_IMAGE is the Docker registry to push to.
  • Once the build is done, run docker push $REGISTRY_IMAGE to push it to the registry.

Procedure (Azure)

For the Azure CLI, the instructions from https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt#option-1-install-with-one-command are followed

  • Create a file called Dockerfile which defines the custom image.
    FROM hashicorp/tfc-agent:latest

    USER root

    RUN curl -L https://aka.ms/InstallAzureCLIDeb | bash

    USER tfc-agent

    # This line will error on build if the Azure CLI is not installed
    RUN az --version
  • While in the same directory as the Dockerfile, Build the image with docker build -t $REGISTRY_IMAGE . where the $REGISTRY_IMAGE is the Docker registry to push to.
  • Once the build is done, run docker push $REGISTRY_IMAGE to push it to the registry.

Procedure (Google Cloud)

For the Google Cloud CLI, the instructions from https://cloud.google.com/sdk/docs/install are followed. This is the only CLI shown here that requires the PATH to be updated as the Google Cloud API doesn't install into the PATH

  • Create a file called Dockerfile which defines the custom image.
    FROM hashicorp/tfc-agent:latest

    USER root

    RUN mkdir -p /opt/gcloud \
        && curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-450.0.0-linux-x86_64.tar.gz \
        && tar -C /opt/gcloud -xf google-cloud-cli-450.0.0-linux-x86_64.tar.gz \
        && /opt/gcloud/google-cloud-sdk/install.sh --quiet

    USER tfc-agent

    ENV PATH="${PATH}:/opt/gcloud/google-cloud-sdk/bin"

    # This line will error on build if the Google Cloud CLI is not installed
    RUN gcloud --version
  • While in the same directory as the Dockerfile, Build the image with docker build -t $REGISTRY_IMAGE . where the $REGISTRY_IMAGE is the Docker registry to push to.
  • Once the build is done, run docker push $REGISTRY_IMAGE to push it to the registry.

Procedure (AWS + Azure + Google Cloud)

  • Create a file called Dockerfile which defines the custom image.
    FROM hashicorp/tfc-agent:latest

    USER root

    RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" \
        && unzip awscliv2.zip \
        && ./aws/install

    RUN curl -L https://aka.ms/InstallAzureCLIDeb | bash

    RUN mkdir -p /opt/gcloud \
    && curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-450.0.0-linux-x86_64.tar.gz \
    && tar -C /opt/gcloud -xf google-cloud-cli-450.0.0-linux-x86_64.tar.gz \
    && /opt/gcloud/google-cloud-sdk/install.sh --quiet

    USER tfc-agent

    ENV PATH="${PATH}:/opt/gcloud/google-cloud-sdk/bin"

    # This line will error on build if the AWS CLI is not installed
    RUN aws --version

    # This line will error on build if the Azure CLI is not installed
    RUN az --version

    # This line will error on build if the Google Cloud CLI is not installed
    RUN gcloud --version
  • While in the same directory as the Dockerfile, Build the image with docker build -t $REGISTRY_IMAGE . where the $REGISTRY_IMAGE is the Docker registry to push to.
  • Once the build is done, run docker push $REGISTRY_IMAGE to push it to the registry.

Articles in this section

See more

Related articles