Why am i seeing "journal: Suppressed xxxxx messages from /system.slice/vault.service" in my logs

Problem

When reviewing my journald logs for vault i notice sections missing and the journal reporting messages have been suppressed for the vault service.

Cause

The error message (or similar) `journal: Suppressed 30995 messages from /system.slice/vault.service` is caused by journald rate limiting configuration. It is a default safeguard so that a particular service does not overwhelm a system by excessive logging. The default is 10000 log entries every 30 seconds meaning that is any more lines than that need to be logged in that time period will be suppressed possibly resulting in missing critical information in an incident scenario.

Solution:

While rate limiting is a safeguard, it can be disabled by the following:

- edit the /etc/systemd/journald.conf file

- set

RateLimitInterval=0
RateLimitBurst=0

- restart the process

$ systemctl restart rsyslog

Alternatively, those values can be tuned if the output is expected to be to verbose

Outcome

All log message should be now be logging to the journal without being suppressed

Additional Information

While this article is focused on journald, other system services may also have rate limiting configured i.e. rsyslog which may also need to be addressed