Importing KV Entries from Consul Enterprise to OSS

Introduction

There may be scenarios where information such as KV entries need to replicate in another cluster for various reasons. This article outlines this scenario through a use case along with steps to achieve this.

Use Case

The kv export command is used to retrieve KV pairs for the given prefix from Consul's KV store, and write a JSON representation to stdout. This can be used with the command "consul kv import" to move entire trees between Consul clusters. However, the resulting output contains information about namespace and partition which are exclusively available in Consul enterprise. Therefore, when attempting to import the KV entries into a Consul cluster that is running on OSS binary, the import fails. The following process outlines the steps to address this issue.

Process

1. Export the desired prefix into a file.
   consul kv export [options] [PREFIX] > kv-dump.json
   
   
2. Remove the namespace and partition reference using jq and store into a new file.
   jq 'map(del(.namespace) | del(.partition))' kv-dump.json > kv-dump-oss.json
   
   
3. Import the new file into the Consul cluster running OSS binary.
   consul kv import @kv-dump-oss.json

Additional Information

• Export KV Entries
• Import KV Entries