Introduction
Vault has an feature for taking the auto-snapshot for raft backend on local filesystem & over the cloud like aws-s3, google-gcs and azure-blob. We can enable multiple auto-snapshot configuration with different storage_type in single cluster. So here we will see how we can configure the auto-snapshot on DR cluster & PR cluster with these storage_type.
Requirement 1) How to enable raft auto-snapshot on DR cluster with different name on PRIMARY & DR Cluster
Sol 1) In PRIMARY & DR replication, if we enable the raft auto-snapshot on PRIMARY cluster then its configuration will not get replicate to DR cluster so for enabling the raft auto-snapshot on DR cluster we have to promote the DR cluster as PRIMARY & then need to enable the raft auto-snapshot configuration on it. We can also give the different name of snapshot in DR configuration if we are using the single storage for both like aws_s3 etc then it will easy to identify that snapshot belongs to which cluster , i.e. PRIMARY or DR.
Below is the configuration for PRIMARY & DR cluster
#PRIMARY Cluster config with aws_s3
root@vaults0:/home/vagrant# vault list sys/storage/raft/snapshot-auto/config/
#No value found at sys/storage/raft/snapshot-auto/config
root@vaults0:/home/vagrant# vault write -f sys/storage/raft/snapshot-auto/config/s3daily \
> aws_access_key_id=ASIAXXXXXXXXXXXXXXXX \
> aws_secret_access_key=TjXXXXXlXXXXXXXXX/XXX+6S0XXXXXXXX+GC/lPc \
> aws_s3_bucket="vaultstorages3"
> aws_session_token="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" \
> aws_s3_disable_tls=false \
> aws_s3_force_path_style=false \
> aws_s3_region="ap-south-1" \
> aws_s3_server_side_encryption=false \
> file_prefix="vault-snapshot-s3-PRIMARY" \
> interval="1m" \
> retain="300" \
> storage_type="aws-s3"
#Success! Data written to: sys/storage/raft/snapshot-auto/config/s3daily
root@vaults0:/home/vagrant# vault read sys/storage/raft/snapshot-auto/config/s3daily
#Key Value
#--- -----
#aws_access_key_id ASIAXXXXXXXXXXXXXXXX
#aws_s3_bucket vaultstorages3
#aws_s3_disable_tls false
#aws_s3_enable_kms false
#aws_s3_endpoint n/a
#aws_s3_force_path_style false
#aws_s3_kms_key n/a
#aws_s3_region ap-south-1
#aws_s3_server_side_encryption false
#aws_secret_access_key TjXXXXXlXXXXXXXXX/XXX+6S0XXXXXXXX+GC/lPc
#aws_session_token XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
#file_prefix vault-snapshot-s3-PRIMARY
#interval 60
#path_prefix n/a
#retain 300
#storage_type aws-s3
root@vaults0:/home/vagrant# vault read sys/storage/raft/snapshot-auto/status/s3daily
#Key Value
#--- -----
#consecutive_errors 0
#last_snapshot_end 2022-09-18T07:06:04Z
#last_snapshot_error n/a
#last_snapshot_start 2022-09-18T07:06:04Z
#last_snapshot_url s3://vaultstorages3/vault-snapshot-s3-PRIMARY-1663487275533784073.snap
#next_snapshot_start 2022-09-18T07:07:04Z
======================================================================================
#DR Cluster config with aws_s3
#Before proceeding with DR cluster we have to promote it to PRIMARY and then we have to configure the auto-snapshot so here I promoted it as PRIMARY
root@vaults2:/home/vagrant# vault list sys/storage/raft/snapshot-auto/config/
#No value found at sys/storage/raft/snapshot-auto/config
root@vaults2:/home/vagrant# vault write -f sys/storage/raft/snapshot-auto/config/s3daily \
> aws_access_key_id=ASIAXXXXXXXXXXXXXXXX \
> aws_secret_access_key=TjXXXXXlXXXXXXXXX/XXX+6S0XXXXXXXX+GC/lPc \
> aws_s3_bucket="vaultstorages3"
> aws_session_token="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" \
> aws_s3_disable_tls=false \
> aws_s3_force_path_style=false \
> aws_s3_region="ap-south-1" \
> aws_s3_server_side_encryption=false \
> file_prefix="vault-snapshot-s3-DR" \
> interval="1m" \
> retain="300" \
> storage_type="aws-s3"
#Success! Data written to: sys/storage/raft/snapshot-auto/config/s3daily
root@vaults2:/home/vagrant# vault read sys/storage/raft/snapshot-auto/config/s3daily
Key Value
--- -----
aws_access_key_id ASIAXXXXXXXXXXXXXXXX
aws_s3_bucket vaultstorages3
aws_s3_disable_tls false
aws_s3_enable_kms false
aws_s3_endpoint n/a
aws_s3_force_path_style false
aws_s3_kms_key n/a
aws_s3_region ap-south-1
aws_s3_server_side_encryption false
aws_secret_access_key TjXXXXXlXXXXXXXXX/XXX+6S0XXXXXXXX+GC/lPc
aws_session_token XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
file_prefix vault-snapshot-s3-DR
interval 60
path_prefix n/a
retain 300
storage_type aws-s3
root@vaults2:/home/vagrant# vault read sys/storage/raft/snapshot-auto/status/s3daily
#Key Value
#--- -----
#consecutive_errors 0
#last_snapshot_end 2022-09-18T07:23:20Z
#last_snapshot_error n/a
#last_snapshot_start 2022-09-18T07:23:19Z
#last_snapshot_url s3://vaultstorages3/vault-snapshot-s3-DR-1663487309251745854.snap
#next_snapshot_start 2022-09-18T07:24:20Z
#snapshot_start 2022-09-18T07:23:20Z
#snapshot_url s3://vaultstorages3/vault-snapshot-s3-DR-1663487309251745854.snap
#Below is the snapshot list in same s3 bucket for PRIMARY & DR naming convention
Requirement 2) How to enable raft auto-snapshot on PR cluster with different name on PRIMARY & PR Cluster
Sol 2) In PRIMARY & PR replication, if we enable the raft auto-snapshot on PRIMARY cluster then its configuration will not get replicate to PR cluster so for enabling the raft auto-snapshot on PR cluster. We need to login to PR Cluster with user that have appropriate permission & then need to enable the raft auto-snapshot configuration on it. We can also give the different name of snapshot in PR configuration with local storage for both clusters, i.e. PRIMARY or PR.
Below is the configuration for PRIMARY & PR cluster
#PRIMARY Cluster config with storage_type local
root@vaults0:/home/vagrant# vault list sys/storage/raft/snapshot-auto/config/
#No value found at sys/storage/raft/snapshot-auto/config
root@vaults0:/home/vagrant# vault write sys/storage/raft/snapshot-auto/config/daily interval=1m retain=7 path_prefix="snapshot-PRIMARY" file_prefix="PRIMARY" storage_type="local" local_max_space=1073741824
#Success! Data written to: sys/storage/raft/snapshot-auto/config/daily
root@vaults0:/snapshot-PRIMARY# vault list sys/storage/raft/snapshot-auto/config/
#Keys
#----
#daily
#Below is the screenshot of snapshot from PRIMARY Cluster
======================================================================================
#PR Cluster config with storage_type local
#Login to PR cluster with user that have appropriate permission
root@vaults1:/home/vagrant# vault list sys/storage/raft/snapshot-auto/config/
#No value found at sys/storage/raft/snapshot-auto/config
root@vaults1:/home/vagrant# vault write sys/storage/raft/snapshot-auto/config/daily interval=1m retain=7 path_prefix="snapshot-PERF" file_prefix="PR" storage_type="local" local_max_space=1073741824
#Success! Data written to: sys/storage/raft/snapshot-auto/config/daily
root@vaults1:/snapshot-PERF# vault list sys/storage/raft/snapshot-auto/config/
#Keys
#----
#daily
#Below is the screenshot of snapshot from PR Cluster
Using the same way, we can also enable raft auto-snapshot on PRIMARY cluster with having DR Cluster & PR cluster to it in same storage with different name of the snapshot so that we easily identify which snapshot belongs to which cluster, refer the screenshot for storage_type=aws_s3
References
https://www.vaultproject.io/api-docs/system/storage/raftautosnapshots
https://www.vaultproject.io/docs/enterprise/automated-integrated-storage-snapshots
All above scenarion has been performed on local lab.