Note: Using a promoted DR to achieve a transition in the seal type is the recommended approach instead of seal migration. Alternatively it's best verify the target seal stanza being migrated onto in an independent Vault instance prior to committing to performing the final migration steps in any live environment.
Introduction
This article specifically addresses the issue of missing the required permissions in an IAM policy for AWS KMS, which can cause a seal migration operation to fail.
Problem
Vault cluster nodes that have already been unsealed, changes back to a sealed state while issuing the command vault operator unseal -migrate.
Here is an excerpt of the operational logs showing the error:
error setting new recovery key information: failed to encrypt keys for storage: error encrypting data: AccessDeniedException: User: arn:aws:iam:xxxxxx:user/xxxxxx is not authorized to perform kms:Encrypt on resource: arn:aws:kms:<region>:xxxxxx:key/xxxxxx because no identity-based policy allows the kms:Encrypt action
Please note the error :
AccessDeniedException: User: arn.... is not authorized to perform kms:Encrypt
Cause
A typical cause can be the missing kms:Encrypt permission on the IAM permissions of the principal that Vault uses or the KMS key policy for the KMS key applicable.
Overview of possible solutions (if applicable)
Solutions:
Add the following set of permissions to the principal used or the KMS key policy:
actions = [
"kms:Encrypt",
"kms:Decrypt",
"kms:DescribeKey",
]
Outcome
With the proper permissions in place Vault should be able to successfully encrypt the master key and the seal migration should complete.