Problem
Terraform Enterprise (TFE) deployed via Helm in AKS across multiple environments fails to forward logs (atlas.log) from the TFE pod to Azure Log Analytics Workspace using Azure Monitor Data Collection Rules (DCR), despite logs being accessible in the container via PVC mount, no logs appear in Azure Monitor under CustomLogs.
Cause
-
The default TFE log forwarding was not enabled, as shown in the support bundle:
"log_forwarding": {
"ConfigData": "",
"config_path": "",
"enabled": false
}
Moreover, the recommended method in TFE documentation relies on enabling log forwarding directly in the pod, which is ephemeral. When pods restart, the forwarding configuration is lost unless manually reapplied. Also, the path (
/var/log/terraform-enterprise
) used for theTFE_LOG_FORWARDING_CONFIG_PATH
shouldn't be used because Fluent Bit configuration files (fluent-bit.conf
, etc.) are static configuration files, not runtime logs.
Solutions:
- To automate log forwarding in all environments and avoid manual intervention :
-
Enable Log Forwarding via Environment Variables in values.yaml:
TFE_LOG_FORWARDING_ENABLED: true
TFE_LOG_FORWARDING_CONFIG_PATH: "/etc/fluent-bit-custom/config"
-
Mount Fluent Bit Config file using a volume:
- type: bind
source: /opt/tfe/fluent-bit.conf
target: /etc/fluent-bit-custom/config
-
Outcome
By enabling TFE_LOG_FORWARDING_ENABLED and properly mounting the Fluent Bit configuration, TFE will automatically forward logs from all pod instances across environments, ensuring centralized visibility in Azure Log Analytics without requiring manual reconfiguration after pod restarts.