Introduction
Consul requires full connectivity between all agents in a datacenter within a LAN gossip pool. However, we could have a requirement to set up a cluster where we can segregate Consul Clients LAN gossip pool to limited/restricted to a single consul network segment.
Using this guide, we are going to configure such a set up where our Consul Clients on EKS cluster can join Consul Servers in particular network segments.
Prerequisites
To achieve this setup, we would need to have the following requirements in place.
- Consul-Helm Chart:- Fo this demo, use
0.49.8helm chart, however you can use any chart version. - Consul Version:- For this demo use
1.13.9+ent, however you could any version that supports network segments. - K8s version:
1.27
Step-by-Step Guide
1. Create 3 Consul Servers hosted on AWS EC2
2. Install Consul Enterprise binary
1.13.9+ent and start consul agent with following configuration file with two segments (alpha and beta mapped on 8303 and 8304 ports respectively for serf LAN gossip).
3. Set below values:-
log_level = "DEBUG"
server = true
license_path = "/etc/consul.d/license.hclic"
bootstrap_expect = 3
datacenter = "dc1"
node_name = "consul-server"
client_addr = "0.0.0.0"
advertise_addr = "172.31.44.51"
retry_join = ["172.31.40.83","172.31.43.55"]
rejoin_after_leave = true
data_dir = "/opt/consul"
ports = {
http = 8500
grpc = 8502
}
ui_config = {
enabled = true
}
acl = {
enabled = true
default_policy = "deny"
enable_token_persistence = true
}
connect {
enabled = true
}
segments = [
{
name = "alpha"
port = 8303
},
{
name = "beta"
port = 8304
}
Create an EKS Cluster with 1/3/5 nodes, and install the following helm chart using
values.yaml to specify Consul Client to join a particular network segment (let's say alpha here for port 8303).
values.yaml
global:
name: consul
datacenter: dc1
acls:
manageSystemACLs: true
createReplicationToken: true
bootstrapToken:
secretName: consul-bootstrap-token
secretKey: token
enterpriseLicense:
secretName: 'consul-license-secret'
secretKey: 'license'
enableConsulNamespaces: true
image: "hashicorp/consul-enterprise:1.13.9-ent"
externalServers:
enabled: true
httpsPort: 8500
hosts: ["172.31.44.51","172.31.40.83","172.31.43.55"] #External Consul Servers VMs IP
#Below is our EKS cluster's public endpoint
k8sAuthMethodHost: https://3CACF42A88C961E49FEB9BB1C786AD82.gr7.ap-south-1.eks.amazonaws.com
server:
enabled: false
connectInject:
enabled: true
controller:
enabled: true
client:
enabled: true
join: ["172.31.44.51:8303","172.31.40.83:8303","172.31.43.55:8303"]
extraConfig: |
{
"segment": "alpha"
}
$ helm install consul hashicorp/consul --values values.yaml --version 0.49.8 --wait --debug
Lastly, we could see the following output upon helm chart installation completion.
$ kubectl get pods -n consul NAME READY STATUS RESTARTS AGE consul-client-d8wv7 1/1 Running 0 43h consul-client-lgr7b 1/1 Running 0 43h consul-client-rplbz 1/1 Running 0 43h consul-connect-injector-59bf65798b-hmvhz 1/1 Running 0 43h consul-webhook-cert-manager-9c6d78cdc-9r46b 1/1 Running 0 43h
We can also validate the consul client
segment status by doing an "ssh" to Consul Server VM. Following command highlights that consul clients have joined the alpha segment.
root@ip-172-31-44-51:/etc/consul.d# consul members Node Address Status Type Build Protocol DC Partition Segment consul-server 172.31.44.51:8301 alive server 1.13.9+ent 2 dc1 default <all> consul-server-2 172.31.40.83:8301 alive server 1.13.9+ent 2 dc1 default <all> consul-server-3 172.31.43.55:8301 alive server 1.13.9+ent 2 dc1 default <all> ip-172-31-31-60.ap-south-1.compute.internal 172.31.16.247:8301 alive client 1.13.9+ent 2 dc1 default alpha ip-172-31-33-131.ap-south-1.compute.internal 172.31.38.105:8301 alive client 1.13.9+ent 2 dc1 default alpha ip-172-31-45-144.ap-south-1.compute.internal 172.31.40.241:8301 alive client 1.13.9+ent 2 dc1 default alpha
Conclusion
Using the above procedure, Network segments enable you to operate a Consul datacenter without full mesh connectivity between agents using a LAN gossip pool. We can create segmented cluster to let consul clients to join respective segment, which improves LAN serf gossip.We can extend the above setup by adding more EKS cluster with consul clients joining other network segments, for example
beta segment.