Introduction
Consul requires full connectivity between all agents in a datacenter within a LAN gossip pool. However, we could have a requirement to set up a cluster where we can segregate Consul Clients LAN gossip pool to limited/restricted to a single consul network segment.
Using this guide, we are going to configure such a set up where our Consul Clients on EKS cluster can join Consul Servers in particular network segments.
Prerequisites
To achieve this setup, we would need to have the following requirements in place.
- Consul-Helm Chart:- Fo this demo, use
0.49.8
helm chart, however you can use any chart version. - Consul Version:- For this demo use
1.13.9+ent
, however you could any version that supports network segments. - K8s version:
1.27
Step-by-Step Guide
1. Create 3 Consul Servers hosted on AWS EC2
2. Install Consul Enterprise binary
1.13.9+ent
and start consul agent with following configuration file with two segments (alpha
and beta
mapped on 8303
and 8304
ports respectively for serf LAN gossip).
3. Set below values:-
log_level = "DEBUG" server = true license_path = "/etc/consul.d/license.hclic" bootstrap_expect = 3 datacenter = "dc1" node_name = "consul-server" client_addr = "0.0.0.0" advertise_addr = "172.31.44.51" retry_join = ["172.31.40.83","172.31.43.55"] rejoin_after_leave = true data_dir = "/opt/consul" ports = { http = 8500 grpc = 8502 } ui_config = { enabled = true } acl = { enabled = true default_policy = "deny" enable_token_persistence = true } connect { enabled = true } segments = [ { name = "alpha" port = 8303 }, { name = "beta" port = 8304 }
Create an EKS Cluster with 1/3/5 nodes, and install the following helm chart using
values.yaml
to specify Consul Client to join a particular network segment (let's say alpha
here for port 8303).
values.yaml
global: name: consul datacenter: dc1 acls: manageSystemACLs: true createReplicationToken: true bootstrapToken: secretName: consul-bootstrap-token secretKey: token enterpriseLicense: secretName: 'consul-license-secret' secretKey: 'license' enableConsulNamespaces: true image: "hashicorp/consul-enterprise:1.13.9-ent" externalServers: enabled: true httpsPort: 8500 hosts: ["172.31.44.51","172.31.40.83","172.31.43.55"] #External Consul Servers VMs IP #Below is our EKS cluster's public endpoint k8sAuthMethodHost: https://3CACF42A88C961E49FEB9BB1C786AD82.gr7.ap-south-1.eks.amazonaws.com server: enabled: false connectInject: enabled: true controller: enabled: true client: enabled: true join: ["172.31.44.51:8303","172.31.40.83:8303","172.31.43.55:8303"] extraConfig: | { "segment": "alpha" }
$ helm install consul hashicorp/consul --values values.yaml --version 0.49.8 --wait --debug
Lastly, we could see the following output upon helm chart installation completion.
$ kubectl get pods -n consul NAME READY STATUS RESTARTS AGE consul-client-d8wv7 1/1 Running 0 43h consul-client-lgr7b 1/1 Running 0 43h consul-client-rplbz 1/1 Running 0 43h consul-connect-injector-59bf65798b-hmvhz 1/1 Running 0 43h consul-webhook-cert-manager-9c6d78cdc-9r46b 1/1 Running 0 43h
We can also validate the consul client
segment
status by doing an "ssh" to Consul Server VM. Following command highlights that consul clients have joined the alpha
segment.
root@ip-172-31-44-51:/etc/consul.d# consul members Node Address Status Type Build Protocol DC Partition Segment consul-server 172.31.44.51:8301 alive server 1.13.9+ent 2 dc1 default <all> consul-server-2 172.31.40.83:8301 alive server 1.13.9+ent 2 dc1 default <all> consul-server-3 172.31.43.55:8301 alive server 1.13.9+ent 2 dc1 default <all> ip-172-31-31-60.ap-south-1.compute.internal 172.31.16.247:8301 alive client 1.13.9+ent 2 dc1 default alpha ip-172-31-33-131.ap-south-1.compute.internal 172.31.38.105:8301 alive client 1.13.9+ent 2 dc1 default alpha ip-172-31-45-144.ap-south-1.compute.internal 172.31.40.241:8301 alive client 1.13.9+ent 2 dc1 default alpha
Conclusion
Using the above procedure, Network segments enable you to operate a Consul datacenter without full mesh connectivity between agents using a LAN gossip pool. We can create segmented cluster to let consul clients to join respective segment, which improves LAN serf gossip.We can extend the above setup by adding more EKS cluster with consul clients joining other network segments, for example
beta
segment.