The information contained in this article has been verified as up-to-date on the date of the original publication of the article. HashiCorp endeavors to keep this information up-to-date and correct, but it makes no representations or warranties of any kind, express or implied, about the ongoing completeness, accuracy, reliability, or suitability of the information provided.
All information contained in this article is for general information purposes only. Any reliance you place on such information as it applies to your use of your HashiCorp product is therefore strictly at your own risk.
NOTE: This will cause WAN traffic to fail during this process, however all traffic inside each cluster will continue to work.
Here are the following steps to remove WAN federation between Consul clusters:
1) Use iptables rules to drop all traffic between the two WAN federated clusters. This will cause both clusters to think nodes in the other cluster have failed.
Try this with root access:
sudo iptables -A OUTPUT -p tcp --dport 8302 -j REJECT
sudo iptables -A INPUT -p tcp --dport 8302 -j REJECT
sudo iptables -A OUTPUT -p udp --dport 8302 -j REJECT
sudo iptables -A INPUT -p udp --dport 8302 -j REJECT
If you’re working remotely via SSH, you might want to add this (-I inserts it before all other rules in INPUT):
iptables -I INPUT -p tcp --dport 22 -j ACCEPT
If your SSH service is listening on another port, you’ll have to use that port instead of 22.
2) After the clusters have been cleanly separated, you'll need to remove retry_join_wan parameter in the configuration file on each consul node accordingly, and reboot each node to update these values.
Example:
retry_join_wan = ["dc2-server-1", "dc2-server-2", "dc2-server-3"]
Note: The value can contain IPv4, IPv6, or DNS addresses.
3) Then you can then use force-leave cli command to cleanly separate the two WAN federated clusters.
Use this command below:
consul force-leave [options] node
If you have ACL's enabled and need to pass a token, use the -token=<value> in the options before specifying the node name.
4) To re-open port 8302 using iptables, use the same command but instead of using REJECT, add ACCEPT:
sudo iptables -A INPUT -p tcp --dports 8302 -j ACCEPT
sudo iptables -A INPUT -p udp --dports 8302 -j ACCEPT
sudo iptables -A OUTPUT -p tcp --dports 8302 -j ACCEPT
sudo iptables -A OUTPUT -p udp --dports 8302 -j ACCEPT
You can also simply remove the rules by using the -D flag rather than the -A flag:
sudo iptables -D INPUT -p tcp --dports 8302 -j ACCEPT
sudo iptables -D INPUT -p udp --dports 8302 -j ACCEPT
sudo iptables -D OUTPUT -p tcp --dports 8302 -j ACCEPT
sudo iptables -D OUTPUT -p udp --dports 8302 -j ACCEPT
Recommendations: It would be a good idea to try this in a sandbox, or testing environment before executing this in production, to make sure this will work proficiently in your environment.