How-to enable and verify Log Forwarding in Terraform Enterprise – HashiCorp Help Center

Introduction

This guide provides instructions on how to enable and verify log forwarding in Terraform Enterprise for various installation types.

Expected Outcome

You will successfully enable log forwarding and confirm that logs are being processed and sent to your configured external destination.

Prerequisites

Terraform Enterprise version v202109-1 or later.
A configured external log destination.

Use Case

This article is for users who are configuring Log Forwarding on Terraform Enterprise and need additional guidance beyond the official logging documentation.

Procedure

First, determine your installation type: Standalone (External Services/Mounted Disk), Active/Active, or Flexible Deployment Options.

Next, create a fluent-bit.conf file containing the [OUTPUT] configuration for one of the supported external destinations.

Follow the steps below that correspond to your installation type.

For Standalone Installations

You can apply these changes using the replicatedctl command-line utility.

Enable log forwarding.

$ replicatedctl app-config set log_forwarding_enabled --value 1

Set the log forwarding configuration from your file.

$ replicatedctl app-config set log_forwarding_config --value "$(cat fluent-bit.conf)"

Apply the configuration. This command restarts Terraform Enterprise to apply the changes.
```
$ replicatedctl app apply-config
```

For Active/Active Installations

For Active/Active installations, you must apply the changes on each node. Drain each node before applying the configuration to ensure no work is interrupted.

Drain the node to stop it from receiving new connections and allow it to complete existing tasks.
```
$ tfe-admin node-drain
```

Enable log forwarding.

$ tfe-admin app-config -k log_forwarding_enabled -v 1

Set the log forwarding configuration from your file.

$ tfe-admin app-config -k log_forwarding_config -v "$(cat fluent-bit.conf)"

Stop the Terraform Enterprise application.
```
$ replicatedctl app stop
```
Confirm the application has stopped successfully.
```
$ replicatedctl app status
```
Start the application to apply the new configuration.
```
$ replicatedctl app start
```
Repeat these steps for the other node(s) in your cluster.

For Flexible Deployment Options Installations

For installations using Flexible Deployment Options (e.g., Docker Compose or Kubernetes), you must update your configuration file and redeploy the container or pod.

Update your deployment configuration (e.g., docker-compose.yaml) with the observability settings.

## docker-compose.yaml settings
## Log Forwarding
TFE_LOG_FORWARDING_CONFIG_PATH: "/container/path/to/fluent-bit.conf"
TFE_LOG_FORWARDING_ENABLED: true

Redeploy your Terraform Enterprise instance to apply the changes. Refer to the Flexible Deployment Options configuration documentation for more details.

Verification

After applying the changes, you can verify that Fluent Bit is processing records by querying its metrics API.

For Docker-based installations, find the IP address of the tfe-fluent-bit container and query the API. The following command uses jq to format the output.
```
$ curl -s http://$(docker inspect tfe-fluent-bit|jq -r '.[].NetworkSettings.Networks[].IPAddress'):2020/api/v1/metrics | jq
```

If jq is not available, you can use this command.

$ curl -s http://$(docker inspect tfe-fluent-bit -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}'):2020/api/v1/metrics

For Flexible Deployment Options, execute the command inside the main tfe container. Note that your container name may differ.
```
$ docker exec -it terraform-enterprise-tfe-1 curl -s http://localhost:2020/api/v1/metrics | jq
```

Example API Output

A successful configuration will show non-zero values for output records, while a failed configuration will show zero records processed.

Fluent Bit API output showing a failed configuration with zero records processed.

Fluent Bit API output showing a successful configuration with a non-zero record count.

Additional Information