Introduction:-
The "/sys/storage/raft/snapshot-auto" endpoints are used to manage automated snapshots with Vault's Raft storage backend.
This article outlines the steps to configure and read the status of automated snapshot configs.
Problem:-
Permission Denied/Identity not found
error is received while trying to read automated snapshots status .
Steps to reproduce the issue
- Create user-managed identity.
- Attach identity to VM and Storage account
- Creating an automated snapshot config in vault.
vault write sys/storage/raft/snapshot-auto/config/hourly interval="1h" retain=0 path_prefix="snapshots/" storage_type=azure-blob azure_container_name="raftbackup" azure_account_name="sasydnpdi1hcv10" azure_auth_mode="managed" azure_client_id="ca23a36b-5dd1-40a6-8be3-bf3efd4a6deb"
Success! Data written to: sys/storage/raft/snapshot-auto/config/hourly
Below error is received while trying to read the status of the config (as an example reading hourly status)
vault read sys/storage/raft/snapshot-auto/status/hourly
Key Value
--- -----
consecutive_errors 0
last_snapshot_end 2024-09-26T03:49:25Z
last_snapshot_error error listing current snapshots: ManagedIdentityCredential authentication failed. ManagedIdentityCredential authentication failed. the requested identity isn't assigned to this resource
GET http://169.254.169.254/metadata/identity/oauth2/token
RESPONSE 400 Bad Request
{
"error": "invalid_request",
"error_description": "Identity not found"
}
To troubleshoot, visit https://aka.ms/azsdk/go/identity/troubleshoot#managed-id
GET http://169.254.169.254/metadata/identity/oauth2/token
RESPONSE 400 Bad Request
{
"error": "invalid_request",
"error_description": "Identity not found"
}
To troubleshoot, visit https://aka.ms/azsdk/go/identity/troubleshoot#managed-id
last_snapshot_start 2024-09-26T03:49:25Z
next_snapshot_start 2024-09-26T04:49:25Z
Error observed in the vault operational logs :
error=
| error listing current snapshots: GET https://vaultblob.blob.core.windows.net/test
| --------------------------------------------------------------------------------
| RESPONSE 403: 403 This request is not authorized to perform this operation using this permission.
| ERROR CODE: AuthorizationPermissionMismatch
| --------------------------------------------------------------------------------
| \ufeffAuthorizationPermissionMismatchThis request is not authorized to perform this operation using this permission. | RequestId:bded56b4-001e-0083-5dcf-0fd4df000000 | Time:2024-09-26T04:52:27.0285106Z
| --------------------------------------------------------------------------------
Cause:-
The storage account doesn't have the required permissions, so the customer is unable to read the status of automated snapshots.
Solution:-
"user-managed" identity should have these role assignments :
- Storage Blob Data Contributor
- Storage Queue Data Contributor
After adding these roles, please test the functionality again in 15 minutes
Steps to add the role assignments :
- Go to Storage account > IAM > Add role assignment.
- In 'Add Role Assignment', select these two roles
- Storage Blob Data Contributor
- Storage Queue Data Contributor
- Assign it to your account.
- These should reflect on your storage account (green highlight)
Outcome :
Successfully able to read automated snapshot status.