Build and Install an Alternative Vault Plugin Version – HashiCorp Help Center

The majority of Vault’s plugins developed by HashiCorp are included in the vault binary itself and so you do not actually require a separate plugin binary to gain the plugin functionality.

An exception to this is the Database Secrets Engine for Oracle plugin, vault-database-plugin-oracle which does not exist in the Vault core source tree, and so is not present in the vault binary; it must be enabled as an external plugin.

What if you wanted to use an alternative build of a plugin that is already contained in the binary, such as the KV Secrets Engine plugin, however?

This guide will present a method for a Vault developer or operator to use alternative plugin builds with Vault, with a brief example using an alternative KV Secrets Engine plugin build.

You can also generalize the steps presented here and apply them to using any Vault plugin build desired; this is a a great way to test new functionality or issue fixes without disturbing the current built-in plugin configuration.

Prerequisites

To follow along, you’ll need a Go development environment, preferably using the same (or possibly higher) Go version that was used to build your Vault binary.

You can determine the go version that built a given vault binary with a tool like delve and steps like these:

$ which vault
/usr/local/bin/vault

$ /usr/local/bin/vault version
Vault v1.1.3+prem ('af0b27edfd5bcd9123b6ac69e90c6e6dfa3654f7')

$ dlv exec /usr/local/bin/vault
Type 'help' for list of commands.
(dlv) b main.main
Breakpoint 1 set at 0x2e2db4f for main.main() /gopath/src/github.com/hashicorp/vault/main.go:9
(dlv) c
> main.main() /gopath/src/github.com/hashicorp/vault/main.go:9 (hits goroutine(1):1 total:1) (PC: 0x2e2db4f)
Warning: debugging optimized function
(dlv) p runtime.buildVersion
"go1.11.5"
(dlv) q

In this case, the Vault version is v1.1.3+prem, built using go version 1.11.5 so we’ll want to use at least that version of Go to build our alternative plugin.

In this example, we are using Go version 1.12.5:

$ go version
go version go1.12.5 darwin/amd64

Build Custom KV Secrets Engine Plugin

Fetch the plugin source:

$ git clone https://github.com/hashicorp/vault-plugin-secrets-kv \
  $GOPATH/src/github.com/hashicorp/vault-plugin-secrets-kv

Customize the plugin source; for this example, the only changes we’ll make is to move the logger invocation, and add a new Info level log line to announce start up of the custom plugin.

$ cd $GOPATH/src/github.com/hashicorp/vault-plugin-secrets-kv
$ git checkout -b custom

Edit cmd/main.go and modify the main function; it will originally appear something like this:

func main() {
    apiClientMeta := &api.PluginAPIClientMeta{}
    flags := apiClientMeta.FlagSet()
    flags.Parse(os.Args[1:])

    tlsConfig := apiClientMeta.GetTLSConfig()
    tlsProviderFunc := api.VaultPluginTLSProvider(tlsConfig)

    err := plugin.Serve(&plugin.ServeOpts{
        BackendFactoryFunc: kv.Factory,
        TLSProviderFunc:    tlsProviderFunc,
    })
    if err != nil {
        logger := hclog.New(&hclog.LoggerOptions{})

        logger.Error("plugin shutting down", "error", err)
        os.Exit(1)
    }
}

We want a version that instantiates logger sooner, and emits a new log message at Info level before running the plugin, like this:

func main() {
    apiClientMeta := &api.PluginAPIClientMeta{}
    flags := apiClientMeta.FlagSet()
    flags.Parse(os.Args[1:])
    logger := hclog.New(&hclog.LoggerOptions{})
    tlsConfig := apiClientMeta.GetTLSConfig()
    tlsProviderFunc := api.VaultPluginTLSProvider(tlsConfig)

    logger.Info("ALT KV Secrets Engine plugin starting up")
    err := plugin.Serve(&plugin.ServeOpts{
        BackendFactoryFunc: kv.Factory,
        TLSProviderFunc:    tlsProviderFunc,
    })
    if err != nil {
        logger.Error("plugin shutting down", "error", err)
        os.Exit(1)
    }
}

This way, we expect our alternative version of the KV Secrets Engine plugin to emit an Info level message stating “ALT KV Secrets Engine plugin starting up” to the Vault operational log when it is started.

Save the changes to cmd/main.go and we’re ready to build the plugin.

Be sure to consult the documentation for the plugin that you are using to determine the specifics for building it. In our case, that’s contained in the vault-plugin-secrets-kv project README.md under Developing.

The plugin has some dependencies which must be prepared for the first time with make bootstrap, so we will execute that first:

$ make bootstrap

For this example, all development and evaluation occurs on our local machine, so we’ll build only for that platform by invoking make dev:

$ make dev

If you want to compile the plugin for all platforms you can just use make.

When the build successfully completes, the alternative vault-plugin-secrets-kv binary for your local platform will be in the bin directory.

If you chose just make instead, check the pkg directory for the binary you require for your platform.

Register the Alternative Secrets Engine Plugin

Get the SHA 256 summary of the plugin binary as we will need it to register the plugin with Vault:

$ shasum -a 256 bin/vault-plugin-secrets-kv
9e79f0cfe1930623a58cea7f36e614b957d99445018ba29eac5768aa211c7be9  bin/vault-plugin-secrets-kv

We need to register the plugin under a different name than the existing KV Secrets Engine plugin; for this example, we will use alt-kv.

First, copy the binary into your plugin_directory and then using the Vault CLI, register the alternative plugin version making sure to use the name alt-kv as shown (you can also choose any name your prefer, except kv):

$ $vault write sys/plugins/catalog/secret/alt-kv \
  sha256="9e79f0cfe1930623a58cea7f36e614b957d99445018ba29eac5768aa211c7be9" \
  command=vault-plugin-secrets-kv
Success! Data written to: sys/plugins/catalog/secret/alt-kv

Use Alternative KV Secrets Engine Plugin

With the plugin now registered, we can check Vault operational log and observe our specially crafted ALT KV Secrets Engine plugin start up message:

2019-06-21T11:52:08.407-0400 [INFO]  ALT KV Secrets Engine plugin starting up: metadata=true

It should otherwise function as a standard KV Secrets Engine, so let’s try some basics with it:

Enable the Secrets Engine:

$ vault secrets enable alt-kv
Success! Enabled the kv secrets engine at: alt-kv/

Write secret to alt-kv:

$ vault kv put alt-kv/foo zip=zap
Success! Data written to: alt-kv/foo

List alt-kv:

$ vault kv list alt-kv/
Keys
----
foo

Read secret from alt-kv:

=== Data ===
Key    Value
---    -----
zip    zap

Use Built-In KV Secrets Engine

Enable KV Secrets engine:

$ vault secrets enable kv
Success! Enabled the kv secrets engine at: kv/

Write secret to kv:

$ vault kv put kv/foo zip=zap
Success! Data written to: kv/foo

List kv:

$ vault kv list kv/
Keys
----
foo

Read secret from kv:

=== Data ===
Key    Value
---    -----
zip    zap

Demonstrate Plugin and Secrets Engine Co-Existence

We can list plugins and confirm that both KV Secrets Plugins are present:

vault plugin list | grep kv
alt-kv
kv

Here is a list of KV Secrets Engines showing our recently enabled kv and alt-kv as well:

$ vault secrets list | grep kv
alt-kv/       alt-kv       alt-kv_e3db34b6       n/a
kv/           kv           kv_9df233e7           n/a

Where to From Here?

You might also be interested in reviewing the Plugin Replacement and Upgrade article which covers how to upgrade or otherwise replace a plugin.

Resources