Problem
We are having this error on version 1.7.0 and later, to fix this we have to pass in the dr_operation_token
Prerequisites (if applicable)
- Two Vault clusters using Integrated Storage (Raft) as storage backend and Disaster Recovery replication enabled
- Vault Enterprise versions 1.7.0 or later
storage/raft (enterprise): Listing of peers is now allowed on DR secondary cluster nodes, as an update operation that takes in DR operation token for authenticating the request.
- vault/CHANGELOG.md
>storage/raft (enterprise): Listing of peers is now allowed on DR secondary cluster nodes, as an update operation that takes in DR operation token for authenticating the request.
`path disabled in replication DR secondary mode`
If we see this error, we would like to take these steps :
To resolve this, We have to pass in the dr_operation_token for this:
vault operator raft list-peers dr_operation_token "DR_OPERATION_TOKEN_HERE"
And to steps to generate the token.
# To read raft configuration
path "sys/storage/raft/configuration" {
capabilities = [ "read", "update" ]To generate the dr operations token on the secondary please follow this doc link
- Start the DR operation token generation process.
vault operator generate-root -dr-token -init -
In order to generate a DR operation token, the following operation must be executed by each unseal key holder.
vault operator generate-root -dr-token \ -nonce=b4738404-0a11-63aa-2cb6-e77dfd96946f \ PRIMARY_UNSEAL_KEY_1 Nonce b4738404-0a11-63aa-2cb6-e77dfd96946f Started true Progress 1/3 Complete false - Once the threshold has been reached, the output contains the encoded DR operation token.
vault operator generate-root -dr-token \ -nonce=b4738404-0a11-63aa-2cb6-e77dfd96946f \ PRIMARY_UNSEAL_KEY_3 Nonce b4738404-0a11-63aa-2cb6-e77dfd96946f Started true Progress 3/3 Complete true Encoded Token djw4BR1iaDUFIBxaAwpiCC1YGhQHHDMf - Decode the generated DR operation token (
Encoded Token).vault operator generate-root -dr-token \ -decode="djw4BR1iaDUFIBxaAwpiCC1YGhQHHDMf" \ -otp="EYHAkPQYvvz93e8iI3pg1maQ" s.3epDv29lsVfc0oZadkjs6qRN - Finally, promote the DR secondary (Cluster B) to become the new primary. The request must pass the DR operation token.
vault write sys/replication/dr/secondary/promote \ dr_operation_token=s.3epDv29lsVfc0oZadkjs6qRN WARNING! The following warnings were returned from Vault: * This cluster is being promoted to a replication primary. Vault will be unavailable for a brief period and will resume service shortly. - DR Operation Token Strategy
- Where DR_OPERATION token can be obtained through the API