The Vault CLI contains a number of convenience features that enable the operator to make quick changes related to backends and tokens. This article explores the concept of quickly changing a problematic backend mount with the vault lease revoke command and its associated -prefix and -force flags.
The Scenario
You are attempting to disable an unnecessary authentication backend named aws-ec2 when you encounter an error:
$ vault auth disable aws-ec2
Error: Error making API request.
URL: DELETE https://vault.example.com/v1/sys/auth/aws-ec2
Code: 400. Errors:
* failed to revoke 'auth/aws-ec2/login/004c7a3e98ee3c9f3ef83651434a0891a81fb7bb' (1 / 1): failed to revoke token: failed to revoke entry: failed to revoke 'database/creds/mariadb-readonly/
03167e19a1ef25c482728a3f3c21cf739d2b6a69' (1 / 1): failed to revoke entry: resp:(*logical.Response)(nil) err:error during revoke: could not find role with name mariadb-readonlyThe issue at play here is a bit complex, but essentially boils down to:
While revoking the tokens associated with the aws-ec2 backend, a token is encountered that is associated with a MariaDB backend token for a role named mariadb-readonly that does not currently exist.
The Solution
One tempting, but not recommended solution would be to simply create a new role named mariadb-readonly for the associated backend, and then try to disable the aws-ec2 backend again — but what if the MariaDB backend is also gone?
In this example, we described the aws-ec2 backend as being unnecessary, so in addition to disabling it, we don’t mind if the tokens associated with it are all forcibly revoked as well. So how do we quickly and forcibly revoke all of the tokens associated with the backend named aws-ec2?
Let’s learn about what vault lease revoke --help has to say about it:
$ vault lease revoke --help
Usage: vault lease revoke [options] ID
Revokes secrets by their lease ID. This command can revoke a single secret
or multiple secrets based on a path-matched prefix.
-prefix
Treat the ID as a prefix instead of an exact lease ID. This can revoke
multiple leases simultaneously. The default is false.
-force
Delete the lease from Vault even if the secret engine revocation fails.
This is meant for recovery situations where the secret in the target
secret engine was manually removed. If this flag is specified, -prefix
is also required. This is aliased as "-f". The default is false.
It appears we can forcibly revoke all tokens associated with a prefix (the backend mount name) by using the -force and -prefix flags, like this:
$ vault lease revoke -prefix=true -force=true auth/aws-ec2/
Warning! Force-removing leases can cause Vault to become out of sync with
secret engines!
Success! Force revoked any leases with prefix: auth/aws-ec2
and the backend can now be successfully disabled:
$ vault auth disable aws-ec2
Success! Disabled the auth method (if it existed) at: aws-ec2/