AWS KMS to AWS KMS Seal Migration

Avatar
Steven Zamborsky
  • Updated

Introduction

Vault has a plethora of seal types that can be used for sealing and unsealing Vault. One of these is AWS KMS. HashiCorp has plenty of documentation regarding the AWS KMS seal, some of which are below.

Additionally, Vault provides a mechanism to migrate your Vault instance from one seal type to another. This has extensive documentation found below:

This article will focus on circumventing the documented limitation of migrating from oneawskmsseal to a differentawskmsseal. As documented:

NOTE: Seal migration from Auto Unseal to Auto Unseal of the same type is supported since Vault 1.6.0. However, there is a current limitation that prevents migrating from AWSKMS to AWSKMS; all other seal migrations of the same type are supported. Seal migration from One Auto Unseal type (AWS KMS) to different Auto Unseal type (HSM, Azure KMS, etc.) is also supported on older versions as well.

Explanation

At a high-level, the fix for this limitation is simple. Rather than migrating directly from anawskmsseal to anotherawskms, you can migrate fromawskmsseal to Shamir seal, then back to (a different) awskmsseal.

Unsupported: awskms -> awskms
Supported: awskms -> Shamir -> awskms

 

This KB article will walk through how to perform this migration. The prerequisites are as follows:

Prerequisites

  • 2 separate AWS KMS keys. For our demonstration, the 2 keys being used are shown below.

    Alias          Key ID
-----          ------
vault-seal-1   3c56b717-b8ee-4d55-a862-2d3a967b6718
vault-seal-2   2374950f-b074-4d8f-9cfe-e0f2ef9ae3e5

  • Vault instance initialized with 1 of the AWS KMS keys

  • A copy of the recovery keys

Walk through

  • Initialize a Vault server using the first KMS key,vault-seal-1. The config for this example looks like the following.
storage "raft" {
  path    = "/opt/vault/data"
  node_id = "ip-172-31-30-51.us-east-2.compute.internal"
}

listener "tcp" {
  address         = "0.0.0.0:8200"
  tls_disable     = true
}

seal "awskms" {
  region     = "us-east-2"
  kms_key_id = "3c56b717-b8ee-4d55-a862-2d3a967b6718"
}

ui              = true
api_addr        = "http://172.31.30.51:8200"
cluster_addr    = "http://ip-172-31-30-51.us-east-2.compute.internal:8201"
log_level       = "trace"
license_path    = "/etc/vault.d/license.hclic"
  • Use the commandvault operator init > init.txtto initialize, being sure to save the output in the init.txt file, as it will include recovery keys.
[ec2-user@ip-172-31-30-51 ~]$ vault operator init > init.txt
[ec2-user@ip-172-31-30-51 ~]$ cat init.txt
Recovery Key 1: M9/TlT07DSqttln1S8hyPgAEQZWoF2DrUdVh3ffdKDm/
Recovery Key 2: y2Q8YzY1AoDQc9t/6CK+BmDo+g4KcxwFZ3ot+xzZjaei
Recovery Key 3: SPJopwWsTXl5X/4GU9LqeG1XNkL6T1uPdG8fR5Wq5SLs
Recovery Key 4: VEAk/wKX3BmM6HSXzlv18loo1FFmw80vY4pjGw6La9r9
Recovery Key 5: RsfIBLKbas0V/4eRe5vFT0hVYjX+NZi56uj3sGUF7F1F

Initial Root Token: hvs.8gZvipS5gwUtpyFpIFqNqgFS

Success! Vault is initialized

Recovery key initialized with 5 key shares and a key threshold of 3. Please
securely distribute the key shares printed above.

[ec2-user@ip-172-31-30-51 ~]$ vault status
Key                      Value
---                      -----
Recovery Seal Type       shamir
Initialized              true
Sealed                   false
Total Recovery Shares    5
Threshold                3
Version                  1.12.0-rc1+ent
Build Date               2022-09-30T10:19:00Z
Storage Type             raft
Cluster Name             vault-cluster-6be93c71
Cluster ID               a1c852f9-0b5e-7af3-a600-edd9abee172f
HA Enabled               true
HA Cluster               https://ip-172-31-30-51.us-east-2.compute.internal:8201
HA Mode                  active
Active Since             2022-10-13T20:06:36.014018739Z
Raft Committed Index     95
Raft Applied Index       95
Last WAL                 27

The Vault cluster is now configured with KMS auto-unseal.

To proceed with migrating it to the AWS KMS key namedvault-seal-2, first migrate this server to Shamir seal.

  • Add disabled = "true"in theawskmsseal stanza in the config file.
  • Stop the Vault service and update the seal stanza as shown below.
seal "awskms" {
  region     = "us-east-2"
  kms_key_id = "3c56b717-b8ee-4d55-a862-2d3a967b6718"
  disabled = "true"
}

After editing the seal stanza and restarting Vault, the following log will be in the Vault operational logs. This confirms that Vault can be migrated fromawskmsto Shamir.

Oct 13 20:11:27 ip-172-31-30-51.us-east-2.compute.internal vault[23371]: 2022-10-13T20:11:27.963Z [WARN]  core: entering seal migration mode; Vault will not automatically unseal even if using an autoseal: from_barrier_type=awskms to_barrier_type=shamir
  • To perform the migration, run the commandvault operator unseal -migrateand enter a recovery key from the earlier step. Repeat this until we meet the required key threshold. After the threshold is met, the migration is complete, and the seal type is now Shamir.
[ec2-user@ip-172-31-30-51 ~]$ vault operator unseal -migrate M9/TlT07DSqttln1S8hyPgAEQZWoF2DrUdVh3ffdKDm/
Key                           Value
---                           -----
Seal Type                     shamir
Initialized                   true
Sealed                        true
Total Shares                  5
Threshold                     3
Unseal Progress               1/3
Unseal Nonce                  da805071-d54f-ef06-6644-352c7c9ff16e
Seal Migration in Progress    true
Version                       1.12.0-rc1+ent
Build Date                    2022-09-30T10:19:00Z
Storage Type                  raft
HA Enabled                    true
[ec2-user@ip-172-31-30-51 ~]$ vault operator unseal -migrate y2Q8YzY1AoDQc9t/6CK+BmDo+g4KcxwFZ3ot+xzZjaei
Key                           Value
---                           -----
Seal Type                     shamir
Initialized                   true
Sealed                        true
Total Shares                  5
Threshold                     3
Unseal Progress               2/3
Unseal Nonce                  da805071-d54f-ef06-6644-352c7c9ff16e
Seal Migration in Progress    true
Version                       1.12.0-rc1+ent
Build Date                    2022-09-30T10:19:00Z
Storage Type                  raft
HA Enabled                    true
[ec2-user@ip-172-31-30-51 ~]$ vault operator unseal -migrate SPJopwWsTXl5X/4GU9LqeG1XNkL6T1uPdG8fR5Wq5SLs
Key                           Value
---                           -----
Seal Type                     shamir
Initialized                   true
Sealed                        false
Total Shares                  5
Threshold                     3
Seal Migration in Progress    true
Version                       1.12.0-rc1+ent
Build Date                    2022-09-30T10:19:00Z
Storage Type                  raft
Cluster Name                  vault-cluster-6be93c71
Cluster ID                    a1c852f9-0b5e-7af3-a600-edd9abee172f
HA Enabled                    true
HA Cluster                    n/a
HA Mode                       standby
Active Node Address           <none>
Raft Committed Index          190
Raft Applied Index            190
[ec2-user@ip-172-31-30-51 ~]$ vault status
Key                     Value
---                     -----
Seal Type               shamir
Initialized             true
Sealed                  false
Total Shares            5
Threshold               3
Version                 1.12.0-rc1+ent
Build Date              2022-09-30T10:19:00Z
Storage Type            raft
Cluster Name            vault-cluster-6be93c71
Cluster ID              a1c852f9-0b5e-7af3-a600-edd9abee172f
HA Enabled              true
HA Cluster              https://ip-172-31-30-51.us-east-2.compute.internal:8201
HA Mode                 active
Active Since            2022-10-13T20:15:54.819946844Z
Raft Committed Index    214
Raft Applied Index      214
Last WAL                30

After the migration, the following log will appear in the Vault operational logs:

Oct 13 20:15:54 ip-172-31-30-51.us-east-2.compute.internal vault[23371]: 2022-10-13T20:15:54.722Z [INFO]  core: seal migration complete
  • Stop the Vault service and edit theawskmsseal stanza again. Remove thedisabled = "true"line and update thekms_key_idto be the key ID ofvault-seal-2. Theawskmsseal stanza should look like the following:
seal "awskms" {
  region     = "us-east-2"
  kms_key_id = "2374950f-b074-4d8f-9cfe-e0f2ef9ae3e5"
}

Restart Vault and note another operational log message regarding another seal migration.

Oct 13 20:26:54 ip-172-31-30-51.us-east-2.compute.internal vault[23518]: 2022-10-13T20:26:54.641Z [WARN]  core: entering seal migration mode; Vault will not automatically unseal even if using an autoseal: from_barrier_type=shamir to_barrier_type=awskms
  • Run thevault operator unseal -migratecommand repeatedly until the key threshold is met and the migration completes.
[ec2-user@ip-172-31-30-51 ~]$ vault operator unseal -migrate M9/TlT07DSqttln1S8hyPgAEQZWoF2DrUdVh3ffdKDm/
Key                           Value
---                           -----
Recovery Seal Type            shamir
Initialized                   true
Sealed                        true
Total Recovery Shares         5
Threshold                     3
Unseal Progress               1/3
Unseal Nonce                  8c5698dd-c3ce-6c47-3a0f-5f04bc826123
Seal Migration in Progress    true
Version                       1.12.0-rc1+ent
Build Date                    2022-09-30T10:19:00Z
Storage Type                  raft
HA Enabled                    true
[ec2-user@ip-172-31-30-51 ~]$ vault operator unseal -migrate y2Q8YzY1AoDQc9t/6CK+BmDo+g4KcxwFZ3ot+xzZjaei
Key                           Value
---                           -----
Recovery Seal Type            shamir
Initialized                   true
Sealed                        true
Total Recovery Shares         5
Threshold                     3
Unseal Progress               2/3
Unseal Nonce                  8c5698dd-c3ce-6c47-3a0f-5f04bc826123
Seal Migration in Progress    true
Version                       1.12.0-rc1+ent
Build Date                    2022-09-30T10:19:00Z
Storage Type                  raft
HA Enabled                    true
[ec2-user@ip-172-31-30-51 ~]$ vault operator unseal -migrate SPJopwWsTXl5X/4GU9LqeG1XNkL6T1uPdG8fR5Wq5SLs
Key                           Value
---                           -----
Recovery Seal Type            shamir
Initialized                   true
Sealed                        false
Total Recovery Shares         5
Threshold                     3
Seal Migration in Progress    true
Version                       1.12.0-rc1+ent
Build Date                    2022-09-30T10:19:00Z
Storage Type                  raft
Cluster Name                  vault-cluster-6be93c71
Cluster ID                    a1c852f9-0b5e-7af3-a600-edd9abee172f
HA Enabled                    true
HA Cluster                    n/a
HA Mode                       standby
Active Node Address           <none>
Raft Committed Index          431
Raft Applied Index            431
[ec2-user@ip-172-31-30-51 ~]$ vault status
Key                      Value
---                      -----
Recovery Seal Type       shamir
Initialized              true
Sealed                   false
Total Recovery Shares    5
Threshold                3
Version                  1.12.0-rc1+ent
Build Date               2022-09-30T10:19:00Z
Storage Type             raft
Cluster Name             vault-cluster-6be93c71
Cluster ID               a1c852f9-0b5e-7af3-a600-edd9abee172f
HA Enabled               true
HA Cluster               https://ip-172-31-30-51.us-east-2.compute.internal:8201
HA Mode                  active
Active Since             2022-10-13T20:28:20.601891415Z
Raft Committed Index     451
Raft Applied Index       451
Last WAL                 33

The following message will appear in the operational logs again.

Oct 13 20:28:20 ip-172-31-30-51.us-east-2.compute.internal vault[23518]: 2022-10-13T20:28:20.479Z [INFO]  core: seal migration complete

Verify that the auto-unseal is working correctly with thevault-seal-2KMS key.

  • In AWS, disablevault-seal-1to ensure that it can't be used for unsealing Vault (this is a validation step, and is not required). Stop the Vault service and restart it. It should unseal itself automatically.
[ec2-user@ip-172-31-30-51 ~]$ sudo systemctl stop vault
[ec2-user@ip-172-31-30-51 ~]$ sudo systemctl start vault
[ec2-user@ip-172-31-30-51 ~]$ vault status
Key                      Value
---                      -----
Recovery Seal Type       shamir
Initialized              true
Sealed                   false
Total Recovery Shares    5
Threshold                3
Version                  1.12.0-rc1+ent
Build Date               2022-09-30T10:19:00Z
Storage Type             raft
Cluster Name             vault-cluster-6be93c71
Cluster ID               a1c852f9-0b5e-7af3-a600-edd9abee172f
HA Enabled               true
HA Cluster               https://ip-172-31-30-51.us-east-2.compute.internal:8201
HA Mode                  active
Active Since             2022-10-13T20:34:45.39358695Z
Raft Committed Index     615
Raft Applied Index       615
Last WAL                 36

This completes the migration between 2 separate AWS KMS keys.

Articles in this section

See more

Related articles