Setting Up Consul Peering Through Mesh Gateways Using KIND Kubernetes Clusters – HashiCorp Help Center

Overview

This KB outlines the procedure to configure Consul control plane peering via Mesh Gateways across two Kubernetes clusters created locally using KIND.

Prerequisites

Docker installed and running. Refer this.
KIND installed. Refer this.
kubectl installed. Refer this.
Helm installed. Refer this.
Valid Consul Enterprise license file (license.hclic). [This is optional. I am using enterprise version of Consul hence using the license here]. Refer this.

Consul Helm chart repository added:

helm repo add hashicorp https://helm.releases.hashicorp.com

Create KIND Clusters

Create two independent KIND clusters, each simulating a separate Consul datacenter:

kind create cluster --name=dc1
kind create cluster --name=dc2

Verify contexts:

kubectl config get-contexts

3. Deploy Consul on Cluster DC1

3.1 Switch to dc1 context

kubectl config use-context kind-dc1

3.2 Create Consul Enterprise license secret

secret=$(cat license.hclic)
kubectl create secret generic consul-ent-license --from-literal="key=${secret}"

3.3 Install Consul using Helm

values-dc1.yaml

Below are sample generic values for the dc1 datacenter
(Replace ports or images as per your testing needs)

global:
  enabled: true
  name: consul
  image: hashicorp/consul-enterprise:1.21.1-ent
  peering:
    enabled: true
  enterpriseLicense:
    secretName: "consul-ent-license"
    secretKey: "key"
    enableLicenseAutoload: true
  imageK8S: hashicorp/consul-k8s-control-plane:1.7.0
  imageConsulDataplane: hashicorp/consul-dataplane:1.7.0
  datacenter: dc1
  tls:
    enabled: true
server:
  enabled: true
  replicas: 1
ui:
  enabled: true
  service:
    type: NodePort
connectInject:
  enabled: true
meshGateway:
  enabled: true
  wanAddress:
    source: NodeIP
    port: 30001
  service:
    type: NodePort
    nodePort: 30001

helm install consul hashicorp/consul --values=values-dc1.yaml

3.4 Deploy Mesh Gateway and Peering Acceptor

kubectl apply -f mesh.yaml
kubectl apply -f acceptor.yaml

mesh.yaml

apiVersion: consul.hashicorp.com/v1alpha1
kind: Mesh
metadata:
  name: mesh
spec:
  peering:
    peerThroughMeshGateways: true

acceptor.yaml

apiVersion: consul.hashicorp.com/v1alpha1
kind: PeeringAcceptor
metadata:
  name: cluster-02 ## The name of the peer you want to connect to
spec:
  peer:
    secret:
      name: "peering-token"
      key: "data"
      backend: "kubernetes"

Once the acceptor is applied, Consul will automatically create a peering-token secret.

3.5 Export the peering token

kubectl get secret peering-token --output yaml > peering-token.yaml

This token will be applied to the second cluster.

4. Deploy Consul on Cluster DC2

4.1 Switch to dc2 context

kubectl config use-context kind-dc2

4.2 Create Consul Enterprise license secret

kubectl create secret generic consul-ent-license --from-literal="key=${secret}"

4.3 Install Consul

values-dc2.yaml

Below are sample generic values for the dc1 datacenter
(Replace ports or images as per your testing needs)

global:
  enabled: true
  name: consul
  image: hashicorp/consul-enterprise:1.21.1-ent
  peering:
    enabled: true
  enterpriseLicense:
    secretName: "consul-ent-license"
    secretKey: "key"
    enableLicenseAutoload: true
  imageK8S: hashicorp/consul-k8s-control-plane:1.7.0
  imageConsulDataplane: hashicorp/consul-dataplane:1.7.0
  datacenter: dc2
  tls:
    enabled: true
server:
  enabled: true
  replicas: 1
ui:
  enabled: true
  service:
    type: NodePort
connectInject:
  enabled: true
meshGateway:
  enabled: true
  wanAddress:
    source: NodeIP
    port: 30002
  service:
    type: NodePort
    nodePort: 30002

helm install consul hashicorp/consul --values=values-dc2.yaml

4.4 Deploy Mesh Gateway and Inject Peering Token

kubectl apply -f mesh.yaml
kubectl apply -f peering-token.yaml

mesh.yaml

apiVersion: consul.hashicorp.com/v1alpha1
kind: Mesh
metadata:
  name: mesh
spec:
  peering:
    peerThroughMeshGateways: true

4.5 Deploy Peering Dialer

kubectl apply -f dialer.yaml

dialer.yaml

apiVersion: consul.hashicorp.com/v1alpha1
kind: PeeringDialer
metadata:
  name: cluster-01 ## The name of the peer you want to connect to
spec:
  peer:
    secret:
      name: "peering-token"
      key: "data"
      backend: "kubernetes"

After applying the dialer manifest, the peering connection will be established.

5. Validating the Peering Connection

5.1 Enter the Consul server pod

Example (cluster dc1):

kubectl exec -it consul-server-0 -- sh

5.2 List all peerings

consul peering list

You should see an entry for the remote cluster.

5.3 Verify peering state via API

From cluster dc2:

curl -k https://127.0.0.1:8501/v1/peering/dc1 | jq

From cluster dc1:

curl -k https://127.0.0.1:8501/v1/peering/dc2 | jq

Expected output includes:

"State": "ACTIVE"

6. References

https://developer.hashicorp.com/consul/docs/deploy/server/k8s/enterprise#create-kubernetes-secret
https://developer.hashicorp.com/consul/docs/east-west/cluster-peering/tech-specs/k8s
https://developer.hashicorp.com/consul/docs/east-west/mesh-gateway/cluster-peer#configuration
https://developer.hashicorp.com/consul/docs/east-west/cluster-peering/establish/k8s
https://developer.hashicorp.com/consul/docs/reference/k8s/helm#h-meshgateway
https://developer.hashicorp.com/consul/docs/east-west/cluster-peering/establish/k8s?page=k8s&page=connect&page=cluster-peering&page=usage&page=establish-peering