The provided scripts in this document are merely referential suggestions and should be extended and or adjusted according to your needs especially in the case namespaces for example.
Leveraging Replication & Snapshots
A few methods can be considered for the purposes of migrating KV secrets from one Vault instance to another.
If Vault Enterprise is in use then one approach can be to use an affiliated PR or DR cluster, which would replicate all authentication as well as secrets mounts and then to proceed with promoting that cluster and subsequently disabling all mounts that are not needed.
Another approach that's similar to the use of a DR / PR cluster may also be by way of a snapshot (via Consul or Integrated Storage / Raft) that can be restored onto the target Vault instance and subsequently all other mounts outside the KV secrets mounts then disabled.
The advantage with either DR / PR or snapshot is that an exact replicas of the original KV secrets engine is achieved; this maintains all metadata and prior revisions of each KV secrets without any loss.
The script below could be run on the newly replicated / snapshot-restored cluster to iteratively disable all secrets and authentication mounts excluding the named KV secrets mount.
#!/usr/bin/env bash
# // Script uses CLI: `jq`
# // set token & address of new Vault instance containing all replicated mounts
export VAULT_TOKEN=... && export VAULT_ADDR=... ;
# // the secrets and authentication mounts to keep.
VSECRETS_MOUNTS=( 'mykv' ) ;
VAUTH_MOUNTS=( 'myuserpass' ) ;
# // get all secret mount paths
VALL_SECRETS=($(vault secrets list -format=json | jq -r 'keys|.[]')) ;
for sX in ${VALL_SECRETS[*]} ; do
# // ignore built in types 'cubbyhole', 'identity' & 'sys'
if [[ ${sX} == "cubbyhole/" ]] || [[ ${sX} == "identity/" ]] || [[ ${sX} == "sys/" ]] ; then continue ; fi ;
bKEEP=false ;
for sW in ${VSECRETS_MOUNTS[*]} ; do
if [[ ${sW} == ${sX} ]] ; then bKEEP=true ; break ; fi ;
done ;
if [[ ${bKEEP} == false ]] ; then vault secrets disable ${sX} ; fi ;
done ;
# // get all auth mount paths
VALL_AUTH=($(vault auth list -format=json | jq -r 'keys|.[]')) ;
for sY in ${VALL_AUTH[*]} ; do
# // ignore built in 'token' mount.
if [[ ${sY} == "token/" ]] ; then continue ; fi ;
bKEEP=false ;
for sZ in ${VAUTH_MOUNTS[*]} ; do
if [[ ${sZ} == ${sY} ]] ; then bKEEP=true ; break ; fi ;
done ;
# // you may need to consider performing prior to disabling auth mounts.
# vault lease revoke --force --prefix ...
if [[ ${bKEEP} == false ]] ; then vault auth disable ${sY} ; fi ;
done ;
Copying KV manually
In the event that neither DR / PR nor snapshots are an option then it's also possible to recursively read and write KV secrets from one Vault instance to another; however using this method you'll only be able to copy KV entries in their current form without any of its metadata or prior revisions.
Below is an example script that could be used for copying either version 1 or version 2 KV secrets from a source to a destination path:
#!/usr/bin/env bash
# // Script uses CLI: `jq` & `curl`.
# // Vault 1 address & KV path
V1_TOKEN='...' ;
V1_ADDR='https://...:8200' ;
V1_KV='kv_source/' ;
# // Vault 2 address & KV path
V2_TOKEN='...' ;
V2_ADDR='https://...:8200' ;
V2_KV='kv_destination/' ;
export VAULT_TOKEN=${V1_TOKEN} && export VAULT_ADDR=${V1_ADDR} ;
# // ^^ for convenience with Vault CLI and V2 values for API / curl
# // check KV version 1 or 2 for source & destination so as to append 'data/' to path.
V2_KV_VER=$(VAULT_ADD=${V2_ADDR} VAULT_TOKEN=${V2_TOKEN} \
vault secrets list -format=json | jq -r ".[\"${V2_KV}\"]|.options.version") ;
if [[ ${V2_KV_VER} == "2" ]] ; then V2_KV+="data/" ; fi ;
function kv_recurse()
{
local V1_KV_LIST_SUB=() ;
V1_KV_LIST_SUB=($(vault kv list -format=json ${V1_KV}/$1 | jq -r '.[]')) ;
for sY in ${V1_KV_LIST_SUB[*]} ; do
if [[ ${sY} == *'/' ]] ; then kv_recurse $1${sY} ;
else
# // read data
KV_DATA=$(vault kv get -format=json ${V1_KV}$1${sY} | jq '.data') ;
# // strip or add 'data' object subject to kv1 or kv2
if [[ ${V2_KV_VER} == "2" && "$(echo ${KV_DATA} | jq '.data')" == "null" ]] ; then
KV_DATA="{\"data\": ${KV_DATA}}" ;
fi ;
if [[ ${V2_KV_VER} == "1" && ! "$(echo ${KV_DATA} | jq '.data')" == "null" ]] ; then
KV_DATA=$(echo ${KV_DATA} | jq '.data') ;
fi ;
# // re-write to new Vault / KV engine
sRESP=$(curl -k -L -X POST -H "X-Vault-Token: ${V2_TOKEN}" -d "${KV_DATA}" \
-o /dev/null -s -w "%{http_code}\n" ${V2_ADDR}/v1/${V2_KV}$1${sY}) ;
if ! [[ ${sRESP} == "200" || ${sRESP} == "204" ]]; then
printf "ERROR: copying: ${V1_KV}$1${sY} to ${V2_ADDR}/v1/${V2_KV}$1${sY}\n" ;
fi ;
fi ;
done ;
}
# // first list all keys on root of KV path & recurse through them.
V1_KV_LIST=($(vault kv list -format=json ${V1_KV} | jq -r '.[]')) ;
for sX in ${V1_KV_LIST[*]} ; do
if [[ ${sX} == *'/' ]] ; then kv_recurse $sX ;
else
# // read data
KV_DATA=$(vault kv get -format=json ${V1_KV}${sX} | jq '.data') ;
# // strip or add 'data' object subject to kv1 or kv2
if [[ ${V2_KV_VER} == "2" && "$(echo ${KV_DATA} | jq '.data')" == "null" ]] ; then
KV_DATA="{\"data\": ${KV_DATA}}" ;
fi ;
if [[ ${V2_KV_VER} == "1" && ! "$(echo ${KV_DATA} | jq '.data')" == "null" ]] ; then
KV_DATA=$(echo ${KV_DATA} | jq '.data') ;
fi ;
# // re-write to new Vault / KV engine
sRESP=$(curl -k -L -X POST -H "X-Vault-Token: ${V2_TOKEN}" -d "${KV_DATA}" -o \
/dev/null -s -w "%{http_code}\n" ${V2_ADDR}/v1/${V2_KV}${sX}) ;
if ! [[ ${sRESP} == "200" || ${sRESP} == "204" ]] ; then
printf "ERROR: copying: ${V1_KV}${sX} to ${V2_ADDR}/v1/${V2_KV}${sX}\n" ;
fi ;
fi ;
done ;