How to monitor Terraform worker containers resource usage in Docker

June 28, 2024 13:22
Updated

Introduction

Expected Outcome

Monitor the resource usage of your Terraform worker docker container with reference to the RUN_ID and WORKSPACE_NAME in Terraform Enterprise

Prerequisites

Terraform Enterprise until release v202305-1. For release v202302-1 or later run_pipeline_mode must be set to legacy.
SSH access to the Terraform Enterprise instance
Permissions to run Docker commands or Sudo access

Procedure

SSH into your Terraform Enterprise instance

Create a worker_mon.sh file with the following content

#!/bin/bash
container=$(docker ps -q -f ancestor=$(/usr/local/bin/replicatedctl app-config export --template '{{.custom_image_tag.Value}}')|xargs)
while true
do
printf "%s %s\n" CONTAINER_ID $container RUN_ID "$(docker exec $container cat -e /env/TFC_RUN_ID) WORKSPACE_NAME $(docker exec $container cat -e /env/TFC_WORKSPACE_NAME)" "$(docker stats $container --no-stream)"
sleep 1
done

Save the file and make it executable
```
chmod +x worker_mon.sh
```
Confirm there are active Runs and Execute the script as follows (use sudo if needed).
```
./worker_mon.sh
```
Example output

See more