How to delete all Kv-v2 secrets under a specific path – HashiCorp Help Center

Introduction

Expected Outcome

There are edge cases where all the secrets stored on a specific path on a Kv-v2 secrets engine mount must be permanently removed. Currently there is no API option to achieve this task and permanently delete all the subkeys and secrets.

This operation involves custom filters and special conditions that could potentially produce the complete loss of all the child secrets and the metadata associated with those objects. Incorrect execution or adding a wrong starting Kv-v2 path could reach to complete loss of data for which the single fallback option is the restore of Vault database.

WARNINGS: DO NOT perform these steps if your cluster is experiencing performance issues due to leases/writes.

Prerequisites

All Vault versions >= 0.10.3

Use Case

Having a secret Kv-v2 mounted and a set of secrets saved under a deep hierarchy structure like:

Enable secrets engine KV and secrets (demo purpose)

vault secrets enable -path=kv kv-v2
vault kv put kv/prod/blue/server/eu hostname=value1 ip=value2
vault kv put kv/prod/blue/server/us hostname=value11 ip=value22
vault kv putkv/prod/blue/server/apj hostname=value111 ip=value222
vault kv put kv/test/blue/server/eu hostname=value91 ip=value92

Vault secrets engine view

|-- kv/ 
|  \-- prod/ 
|     \-- blue/ 
|        \-- server/ 
|           |-- apj hostname=value111 ip=value222 
|           |-- eu hostname=value1 ip=value2 
|           |-- us hostname=value11 ip=value22 
|  \-- test/ 
|     \-- blue/ 
|        \-- server/ 
|           |-- eu hostname=value91 ip=value92

the requirement is to permanently delete the prefixed kv/prod for setting the environment ready for testing.

Procedure

After creating a consistent backup of the Vault data execute a delete using as argument the path to be permanently deleted.

Step 1 - Perform a backup using the guide from Vault SOP and make sure that you save the generated output on a safe location.
Step 2 - Save a script on a Linux/MAC OS console that is having the necessary permissions to access the Vault cluster. The script may be named: delete_secrets.sh

#!/bin/bash

function delete_key()
{
IFS=$'\n' values=($(vault kv list -format=json "$1" | jq -r '.[]'))

if [ -z $values ]; then
  versions=$(vault kv metadata get -format=json $1 |jq '.data.versions | keys_unsorted[] | tonumber' | tr '\n' ',')
  printf '\n%s \n\t%s' "Deleting secret $1" "with identified versions: $versions"
  printf '\n%s\n' "$(vault kv metadata delete $1)"
else
  :
fi
return
}

function traverse_delete {
  local -r path="$1"
  result=$(vault kv list -format=json $path 2>&1)

  status=$?
  if [ ! $status -eq 0 ]
  then
    if [[ $result =~ "permission denied" ]]; then
      return
    else
       :
    fi
  >&2 && [ ! "Z$result" == "Z{}" ] && echo "$result"
  else
    :
  fi

for secret in $(echo "$result" | jq -r '.[]'); do
 if [[ "$secret" == */ ]]; then
    traverse_delete "$path$secret"
 else
   printf '%s\t%s' "Found secret:" "$path$secret"
   delete_key "$path$secret"
 fi
 done
}

# MAIN Iterate on all kv secrets or start from the path provided by the input ARG[1]
if [[ "$1" ]]; then
  # Make sure the path always ends with '/'
  search_path=("${1%"/"}/")
else
  :
fi

printf '\n%s: %s\n' "### DELETE ALL the secrets on path" "$search_path"
for _s_path in $search_path; do
  traverse_delete ${_s_path}
done
printf '\n'

Step 3 - Login to the Vault server using a TOKEN with the correct permissions to perform the recursive delete.
Step 4 - Add the below code into the script delete_secret.sh
Step 5 - Execute the script for the specific path. The path for the secret must contain the secrets engine Kv-v2 mount point!
As example, if the secret engine of type Kv-v2 is enabled at path kv and the secret is stored as prod , then the following table applies.

Operation

Details

Execution

bash delete_secrets.sh kv/prod/

Output

### DELETE ALL the secrets on path: kv/
Found secret: kv/TEST/_keep_structure
Deleting secret kv/TEST/_keep_structure
with identified versions: 1,
Success! Data deleted (if it existed) at: kv/metadata/TEST/_keep_structure
Found secret: kv/prod/blue/server/apj
Deleting secret kv/prod/blue/server/apj
with identified versions: 1,
Success! Data deleted (if it existed) at: kv/metadata/prod/blue/server/apj
Found secret: kv/prod/blue/server/eu
Deleting secret kv/prod/blue/server/eu
with identified versions: 1,
Success! Data deleted (if it existed) at: kv/metadata/prod/blue/server/eu
Found secret: kv/prod/blue/server/us
Deleting secret kv/prod/blue/server/us
with identified versions: 1,
Success! Data deleted (if it existed) at: kv/metadata/prod/blue/server/us

Additional Information