Introduction
Problem
You receive the following error after upgrading DR Secondary to DR Primary and performing a lookup:
Error reading secret/path/to/storedsecret: Error making API request.
URL: GET https://vault-dev.domain.net/v1/secret/test/replication/migration
Code: 412. Errors:
* required index state not present
Prerequisites (if applicable)
- Applies to Vault 1.10.5 or earlier
Cause
- Bug: "The issue is that the SSCT Generation counter was not being loaded from storage, so when it was written to storage during DR promotion it would always be 1. The fix is to load the value from storage during DR promotion. DRs never use the generation counter, but on DR clusters the value in memory will always be 1 less than the value in storage. However, the moment the DR gets updated to a primary, the value from storage will be incremented and written back."
Overview of possible solutions (if applicable)
Solutions:
-
A Bug fix has been created in Vault 1.10.7, 1.11.4, and again in 1.12.0.
Outcome
After upgrade this issue is remedied. A potential work around is to perform a rolling restart of the new DR Primary cluster.
Additional Information
-
Reference the Vault Changelog for all bug fixes: https://github.com/hashicorp/vault/blob/main/CHANGELOG.md