Introduction
When configuring automated snapshots a
retain parameter is available which defines how many snapshots are to be kept. When writing a snapshot, if there are more snapshots already stored, the oldest ones will be deleted. When using the AWS-S3 cloud storage type, the path_prefix can affect the retain parameter set.
Scenario
For cloud storage types, the
path_prefix defines the bucket prefix that will be used and for AWS-S3 the trailing / is optional. A leading / however can affect the retain parameter, causing snapshots to retain in the defined directory. If snapshots are not being automatically removed with a retain parameter is set, verify the below configurations.
Procedure
Validate the current automated snapshot configuration:
$ vault read sys/storage/raft/snapshot-auto/config/config1
Key Value
--- -----
aws_access_key_id AKI...
aws_s3_bucket bucket
aws_s3_disable_tls false
aws_s3_enable_kms false
aws_s3_endpoint n/a
aws_s3_force_path_style false
aws_s3_kms_key n/a
aws_s3_region us-east-2
aws_s3_server_side_encryption false
aws_secret_access_key x7LK...
aws_session_token n/a
file_prefix vault-snapshot
interval 120
path_prefix /vault/snapshots/
retain 7
storage_type aws-s3
Note the
path_prefix is configured for /vault/snapshots/Update the path_prefix parameter removing the leading /:
$ vault write sys/storage/raft/snapshot-auto/config/config1 aws_s3_bucket="bucket" aws_s3_region="us-east-2" path_prefix="vault/snapshots/" storage_type="aws-s3"
Success! Data written to: sys/storage/raft/snapshot-auto/config/test
Once complete, the Vault operational logs will show the number of deleted snapshots in order to adhere to the retain parameter set. Log entries will look similar to the below example:
2022-05-11T07:48:28.318-0400 [DEBUG] core.snapshotmgr.config1: snapshot complete: name=config1 elapsed=781.864649ms size=0
2022-05-11T07:48:28.490-0400 [DEBUG] core.snapshotmgr.config1: deleted old snapshots: retain=7 deleted=13 name=config1
Subsequent automated snapshots will log something like the following:
2022-05-11T07:48:58.626-0400 [INFO] core.snapshotmgr.config1: taking auto snapshot
2022-05-11T07:48:59.014-0400 [INFO] storage.raft: starting snapshot up to: index=3698
2022-05-11T07:48:59.054-0400 [INFO] storage.raft: snapshot complete up to: index=3698
2022-05-11T07:48:59.468-0400 [DEBUG] core.snapshotmgr.config1: snapshot complete: name=config1 elapsed=842.562155ms size=0
2022-05-11T07:48:59.714-0400 [DEBUG] core.snapshotmgr.config1: deleted old snapshots: retain=7 deleted=1 name=config1