No logs received from TFE log forwarding – HashiCorp Help Center

Problem

Logs forwarding is configured and enabled, but the logs are not being forwarded to the target system as intended. Additionally, container logs for the log forwarding container (tfe-fluent-bit) reveal no errors.

Prerequisites (if applicable)

Terraform Enterprise, configured with log forwarding enabled
Syslog

Cause

The log_forwarding_config has an invalid configuration for the syslog_message_key.

name syslog
match *
host syslog.logserver.com
port 514
mode udp
syslog_format rfc5424
syslog_maxsize 2048
syslog_hostname_key hostname
syslog_appname_key appname
syslog_procid_key procid
syslog_msgid_key msgid 
syslog_sd_key uls@0
syslog_message_key log

Overview of possible solutions (if applicable)

Solutions:

1. Review the current log_forwarding_configby running the following command:

replicatedctl app-config export --template '{{.log_forwarding_config.Value}}'

[OUTPUT]
    Name                 syslog
    Match                *
    host                 syslog.logserver.com
    port                 514
    mode                 udp
    syslog_message_key   log
    syslog_severity_key  PRIORITY
    syslog_hostname_key  _HOSTNAME
    syslog_appname_key   SYSLOG_IDENTIFIER
    syslog_procid_key    _PID

2. Update the fluent-bit.conf file ensuring that thesyslog_message_key value is set to message:

fluent-bit.conf

[OUTPUT]
    Name                 syslog
    Match                *
    host                 syslog.logserver.com
    port                 514
    mode                 udp
    syslog_message_key   message
    syslog_severity_key  PRIORITY
    syslog_hostname_key  _HOSTNAME
    syslog_appname_key   SYSLOG_IDENTIFIER
    syslog_procid_key    _PID

3. Update the log_forwarding_config by running the following command on each node:

 replicatedctl app-config set log_forwarding_config --value "$(cat fluent-bit.conf)"

4. Restart the application for the changes to take effect.

replicatedctl app stop
replicatedctl app start

Outcome

Logs will be forwarded as intended to the syslog host. If issues persist, please contact HashiCorp Support for further assistance.

Additional Information

Syslog Configuration
https://docs.fluentbit.io/manual/pipeline/outputs/syslog