Issue
Performance Replication has stopped on the Performance Secondary.
In addition, these two errors appear in the Vault operational logs:
[ERROR] core: failed to enable replicated audit backend: mount=file/ error="sanity check failed; unable to open \"/path/to/audit-log\" for writing: open /path/to/audit-log: permission denied"
[ERROR] replication: failed to invalidate key, suspending replication: key=core/audit error="sanity check failed; unable to open \"/path/to/audit-log\" for writing: open /path/to/audit-log: permission denied"
Perquisites
- Vault Enterprise
- Performance Replication
- Enabled audit device(s)
Cause
- A new audit log was enabled on the Performance Primary but the Vault user process did not have sufficient permissions to write to the file.
- The new audit log was replicated to the Performance Secondary cluster.
- Since the Performance Secondary could not write to the audit log, replication stopped working.
- This is considered a blocked audit device and Vault will not be able to complete any requests until the blocked audit device can write.
Solutions
- Grant the Vault user permissions to write to the audit log.
Outcome
- Once write permissions are set for the Vault user, Vault will be able to write again and replication will once again be successful.
Additional Info
- It is recommended that Vault is configured to use multiple audit devices.
https://developer.hashicorp.com/vault/docs/audit
https://developer.hashicorp.com/vault/docs/enterprise/replication