Getting broken pipe error when migrating Vault storage backend from Consul to Raft/Integrated Storage

Problem

The following Error is displayed in Vault Operational Logs when migrating from Consul to Raft Storage Backend for your Vault environment:

storage migration check error: error="Get "https://x.x.x.x:8501/v1/kv/vault/core/migration": write tcp x.x.x.x:xxxxx->x.x.x.x:8501: write: broken pipe"

Prerequisites 

• You are performing storage migration from Consul to Raft.
• You are observing the error broken pipe in Vault Operational log when running the storage migration.

Cause

• The reason of this error is due to Vault unable to reach out to its backend storage during migration because of a certificate/TLS configuration issue.

Solutions:

• First, please ensure that Vault is currently able to connect to Consul backend storage based on its current configuration prior initiating the storage migration. Especially check the certificates configured in your Vault and Consul environment.

• From the storage migration tutorial, it specifically mentioned that for the migration configuration file the "storage_source stanza should be the current storage backend (consul) configuration". Double check and ensure that you have copied the current Vault configuration's consul storage backend to the migration configuration file. Ensure that the certificate is accessible and that the path is correct. You may use tools such as diff to check if there are any formatting/spacing issues that may have been introduced during copying and pasting.
• Alternatively, you can explicitly set verify_incoming to false in your Consul Agent Configuration so that Consul does not verify a client's authenticity. However, note that this option can introduce security issues and please evaluate before setting the config.

Additional Information

• Please refer to the storage migration tutorial and ensure that the steps have followed, in addition to double check the configuration.