Introduction
Problem
When Vault Seal High Availability is enabled and multiple seals are enabled, running vault read -format=json /sys/seal-backend-status might under certain circumstances not display the unhealthy seal(s).
Prerequisites (if applicable)
- Vault Enterprise 1.19.x or earlier versions
- Seal High Availability enabled
- Multiple Seal backends enabled.
Cause
For example in this scenario the vault Operational Logs contains:
May 30 10:41:54 vault[122073]: 2025-05-30T10:41:54.033Z [WARN] cannot determine if seal wrapped entry needs update: there were errors determining the key IDs for one or more seals May 30 10:41:54 vault[122073]: 2025-05-30T10:41:54.033Z [DEBUG] cannot determine if seal wrapped entry needs update: May 30 10:41:54 vault[122073]: err= May 30 10:41:54 vault[122073]: | error refreshing key IDs of Access wrappers May 30 10:41:54 vault[122073]: | error decrypting using seal *************************: seal is unhealthy May 30 10:41:54 vault[122073]:
Running vault read -format=json /sys/seal-backend-statusdisplays:
vault read -format=json /sys/seal-backend-status
{
"request_id": "",
"lease_id": "",
"lease_duration": 0,
"renewable": false,
"data": {
"backends": [
{
"healthy": true,
"name": "azurekeyvault"
},
{
"healthy": false,
"name": "azurekeyvaultsecondary",
"unhealthy_since": "2025-05-30 10:41:53.502248906 +0000 UTC m=+0.555390122"
}
],
"fully_wrapped": false,
"healthy": false,
"unhealthy_since": "2025-05-30 10:41:53.502248906 +0000 UTC m=+0.555390122"
},
"warnings": null
}However after a restart of Vault, vault read -format=json /sys/seal-backend-statusdisplays:
vault read -format=json /sys/seal-backend-status
{
"request_id": "",
"lease_id": "",
"lease_duration": 0,
"renewable": false,
"data": {
"backends": [
{
"healthy": true,
"name": "azurekeyvault"
}
],
"fully_wrapped": false,
"healthy": false
},
"warnings": null
}The fact that vault read -format=json /sys/seal-backend-status shows "fully_wrapped": false and "healthy": false are an indication that the seal configuration is unhealthy, looking at the Vault Operational Logs should provide more insight regarding the possible causes for the issue.
Despite the fact that only one seal is displayed, attempts to invoke the /sys/sealwrap/rewrap api fail with error as displayed below:
vault write -f /sys/sealwrap/rewrap Error writing data to sys/sealwrap/rewrap: Error making API request. URL: PUT http://127.0.0.1:8200/v1/sys/sealwrap/rewrap Code: 500. Errors: * 1 error occurred: * seal health check failed: unhealthy seals: azurekeyvaultsecondary
Solutions:
Running vault read -field=seals -format=json sys/config/state/sanitized should show all configured seals, but won't show their respective statuses:
vault read -field=seals -format=json sys/config/state/sanitized
[
{
"disabled": false,
"name": "azurekeyvault",
"priority": 1,
"type": "azurekeyvault"
},
{
"disabled": false,
"name": "azurekeyvaultsecondary",
"priority": 2,
"type": "azurekeyvault"
}
]Further troubleshooting steps depend on the reasons for the affected seal to be unhealthy, either addressing the issue at hand or removing the affected seal from the Vault Configuration file might be valid steps to resolve the issue. Please note that removing seals from the Vault Configuration file should only be considered in cases where Seal High Availability is configured and multiple seals are configured. When uncertain, reach out to HashiCorp Global Support prior to making any changes.
Additional Information
- Vault Documentation: Configure high availability for Vault seals
- Vault API: sys/seal-backend-status
- Vault API: /sys/config/state
-
Vault API: /sys/sealwrap/rewrap